Sep 11 11:34:19.461930 ip-10-0-135-16 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 11:34:19.865616 ip-10-0-135-16 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 11:34:19.865616 ip-10-0-135-16 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 11:34:19.865616 ip-10-0-135-16 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 11:34:19.865616 ip-10-0-135-16 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 11:34:19.865616 ip-10-0-135-16 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 11:34:19.866291 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.865715    2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 11:34:19.868598 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868582    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:34:19.868598 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868598    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:34:19.868684 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868603    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:34:19.868684 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868606    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:34:19.868684 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868609    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:34:19.868684 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868611    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:34:19.868684 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868614    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:34:19.868684 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868617    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:34:19.868684 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868620    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:34:19.868684 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868623    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:34:19.868684 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868625    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:34:19.868684 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868648    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:34:19.868684 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868651    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:34:19.868684 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868654    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:34:19.868684 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868657    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:34:19.868684 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868659    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:34:19.868684 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868662    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:34:19.868684 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868665    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:34:19.868684 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868669    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:34:19.868684 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868673    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:34:19.868684 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868676    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:34:19.869137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868679    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:34:19.869137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868682    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:34:19.869137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868684    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:34:19.869137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868687    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:34:19.869137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868689    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:34:19.869137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868692    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:34:19.869137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868695    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:34:19.869137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868697    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:34:19.869137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868700    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:34:19.869137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868703    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:34:19.869137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868705    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:34:19.869137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868708    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:34:19.869137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868711    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:34:19.869137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868713    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:34:19.869137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868715    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:34:19.869137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868718    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:34:19.869137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868722    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:34:19.869137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868726    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:34:19.869137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868729    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:34:19.869647 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868732    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:34:19.869647 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868734    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:34:19.869647 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868737    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:34:19.869647 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868739    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:34:19.869647 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868742    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:34:19.869647 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868745    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:34:19.869647 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868747    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:34:19.869647 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868750    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:34:19.869647 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868753    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:34:19.869647 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868755    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:34:19.869647 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868757    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:34:19.869647 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868760    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:34:19.869647 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868762    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:34:19.869647 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868765    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:34:19.869647 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868768    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:34:19.869647 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868770    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:34:19.869647 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868773    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:34:19.869647 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868775    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:34:19.869647 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868778    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:34:19.869647 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868780    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:34:19.870137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868785    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:34:19.870137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868788    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:34:19.870137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868791    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:34:19.870137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868794    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:34:19.870137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868797    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:34:19.870137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868800    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:34:19.870137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868803    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:34:19.870137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868805    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:34:19.870137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868808    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:34:19.870137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868810    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:34:19.870137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868813    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:34:19.870137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868816    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:34:19.870137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868818    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:34:19.870137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868821    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:34:19.870137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868823    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:34:19.870137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868826    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:34:19.870137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868829    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:34:19.870137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868831    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:34:19.870137 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868833    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:34:19.870613 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868836    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:34:19.870613 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868838    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:34:19.870613 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868840    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:34:19.870613 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868843    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:34:19.870613 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868846    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:34:19.870613 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868849    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:34:19.870613 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868851    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:34:19.870613 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868854    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:34:19.870613 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868856    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:34:19.870613 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868859    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:34:19.870613 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868861    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:34:19.870613 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868864    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:34:19.870613 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868866    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:34:19.870613 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868868    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:34:19.870613 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868871    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:34:19.870613 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868873    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:34:19.870613 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868875    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:34:19.870613 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868878    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:34:19.870613 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868880    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:34:19.870613 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.868882    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:34:19.871129 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869564    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:34:19.871129 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869571    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:34:19.871129 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869574    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:34:19.871129 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869577    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:34:19.871129 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869580    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:34:19.871129 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869584    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:34:19.871129 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869586    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:34:19.871129 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869591    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:34:19.871129 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869595    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:34:19.871129 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869598    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:34:19.871129 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869601    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:34:19.871129 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869603    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:34:19.871129 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869606    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:34:19.871129 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869608    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:34:19.871129 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869611    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:34:19.871129 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869614    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:34:19.871129 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869616    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:34:19.871129 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869619    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:34:19.871129 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869621    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:34:19.871129 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869624    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:34:19.871624 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869627    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:34:19.871624 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869644    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:34:19.871624 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869647    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:34:19.871624 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869662    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:34:19.871624 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869676    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:34:19.871624 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869679    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:34:19.871624 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869682    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:34:19.871624 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869685    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:34:19.871624 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869688    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:34:19.871624 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869690    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:34:19.871624 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869693    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:34:19.871624 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869696    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:34:19.871624 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869698    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:34:19.871624 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869701    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:34:19.871624 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869703    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:34:19.871624 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869705    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:34:19.871624 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869708    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:34:19.871624 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869711    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:34:19.871624 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869714    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:34:19.871624 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869716    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:34:19.872171 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869719    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:34:19.872171 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869721    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:34:19.872171 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869725    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:34:19.872171 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869728    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:34:19.872171 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869730    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:34:19.872171 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869734    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:34:19.872171 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869737    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:34:19.872171 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869741    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:34:19.872171 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869743    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:34:19.872171 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869746    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:34:19.872171 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869748    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:34:19.872171 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869750    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:34:19.872171 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869754    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:34:19.872171 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869756    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:34:19.872171 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869759    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:34:19.872171 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869762    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:34:19.872171 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869764    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:34:19.872171 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869767    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:34:19.872171 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869769    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:34:19.872659 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869772    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:34:19.872659 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869774    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:34:19.872659 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869776    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:34:19.872659 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869780    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:34:19.872659 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869783    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:34:19.872659 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869786    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:34:19.872659 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869789    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:34:19.872659 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869791    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:34:19.872659 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869794    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:34:19.872659 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869797    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:34:19.872659 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869799    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:34:19.872659 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869802    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:34:19.872659 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869806    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:34:19.872659 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869809    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:34:19.872659 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869811    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:34:19.872659 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869815    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:34:19.872659 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869817    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:34:19.872659 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869820    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:34:19.872659 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869822    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:34:19.873131 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869825    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:34:19.873131 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869828    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:34:19.873131 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869830    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:34:19.873131 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869833    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:34:19.873131 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869835    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:34:19.873131 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869837    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:34:19.873131 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869840    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:34:19.873131 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869843    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:34:19.873131 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869845    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:34:19.873131 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869848    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:34:19.873131 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869850    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:34:19.873131 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869853    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:34:19.873131 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869855    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:34:19.873131 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869858    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:34:19.873131 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869860    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:34:19.873131 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869863    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:34:19.873131 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869865    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:34:19.873131 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869868    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:34:19.873131 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869870    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:34:19.873131 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869873    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:34:19.873621 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.869875    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:34:19.873621 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870231    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:34:19.873621 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870235    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:34:19.873621 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870238    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:34:19.873621 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870241    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:34:19.873621 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870244    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:34:19.873621 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870247    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:34:19.873621 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870249    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:34:19.873621 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870252    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:34:19.873621 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870255    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:34:19.873621 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870258    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:34:19.873621 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870260    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:34:19.873621 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870263    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:34:19.873621 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870266    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:34:19.873621 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870269    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:34:19.873621 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870271    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:34:19.873621 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870274    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:34:19.873621 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870276    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:34:19.873621 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870279    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:34:19.873621 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870282    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:34:19.874121 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870285    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:34:19.874121 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870287    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:34:19.874121 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870290    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:34:19.874121 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870292    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:34:19.874121 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870295    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:34:19.874121 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870297    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:34:19.874121 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870300    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:34:19.874121 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870304    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:34:19.874121 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870307    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:34:19.874121 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870310    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:34:19.874121 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870313    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:34:19.874121 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870316    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:34:19.874121 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870319    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:34:19.874121 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870321    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:34:19.874121 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870324    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:34:19.874121 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870326    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:34:19.874121 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870329    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:34:19.874121 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870331    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:34:19.874121 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870334    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:34:19.874580 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870337    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:34:19.874580 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870339    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:34:19.874580 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870342    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:34:19.874580 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870344    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:34:19.874580 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870346    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:34:19.874580 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870349    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:34:19.874580 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870352    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:34:19.874580 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870354    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:34:19.874580 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870357    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:34:19.874580 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870360    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:34:19.874580 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870362    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:34:19.874580 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870365    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:34:19.874580 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870367    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:34:19.874580 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870370    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:34:19.874580 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870373    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:34:19.874580 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870375    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:34:19.874580 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870378    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:34:19.874580 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870381    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:34:19.874580 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870383    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:34:19.874580 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870385    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:34:19.875097 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870388    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:34:19.875097 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870391    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:34:19.875097 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870393    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:34:19.875097 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870395    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:34:19.875097 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870398    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:34:19.875097 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870400    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:34:19.875097 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870402    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:34:19.875097 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870405    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:34:19.875097 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870407    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:34:19.875097 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870409    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:34:19.875097 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870412    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:34:19.875097 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870414    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:34:19.875097 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870417    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:34:19.875097 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870420    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:34:19.875097 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870424    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:34:19.875097 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870427    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:34:19.875097 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870430    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:34:19.875097 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870435    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:34:19.875097 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870438    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:34:19.875097 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870441    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:34:19.875579 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870443    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:34:19.875579 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870446    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:34:19.875579 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870449    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:34:19.875579 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870452    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:34:19.875579 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870454    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:34:19.875579 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870457    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:34:19.875579 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870459    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:34:19.875579 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870461    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:34:19.875579 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870464    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:34:19.875579 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870467    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:34:19.875579 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870470    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:34:19.875579 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870472    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:34:19.875579 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870475    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:34:19.875579 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870477    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:34:19.875579 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870480    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:34:19.875579 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870482    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:34:19.875579 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870484    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:34:19.875579 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870487    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:34:19.875579 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870489    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:34:19.875579 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870493    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.870497    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870571    2575 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870607    2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870612    2575 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870617    2575 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870622    2575 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870625    2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870656    2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870663    2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870671    2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870674    2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870677    2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870681    2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870684    2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870687    2575 flags.go:64] FLAG: --cgroup-root=""
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870690    2575 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870693    2575 flags.go:64] FLAG: --client-ca-file=""
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870695    2575 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870698    2575 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870703    2575 flags.go:64] FLAG: --cluster-domain=""
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870706    2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870709    2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870712    2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 11:34:19.876171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870715    2575 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870719    2575 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870722    2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870725    2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870727    2575 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870730    2575 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870733    2575 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870736    2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870739    2575 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870742    2575 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870747    2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870749    2575 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870752    2575 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870755    2575 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870758    2575 flags.go:64] FLAG: --enable-server="true"
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870762    2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870766    2575 flags.go:64] FLAG: --event-burst="100"
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870769    2575 flags.go:64] FLAG: --event-qps="50"
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870773    2575 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870776    2575 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870779    2575 flags.go:64] FLAG: --eviction-hard=""
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870782    2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870785    2575 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870788    2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870791    2575 flags.go:64] FLAG: --eviction-soft=""
Sep 11 11:34:19.876784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870793    2575 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 11:34:19.877440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870796    2575 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 11:34:19.877440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870799    2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 11:34:19.877440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870802    2575 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 11:34:19.877440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870804    2575 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 11:34:19.877440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870807    2575 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 11:34:19.877440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870810    2575 flags.go:64] FLAG: --feature-gates=""
Sep 11 11:34:19.877440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870814    2575 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 11:34:19.877440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870817    2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 11:34:19.877440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870819    2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 11:34:19.877440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870822    2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 11:34:19.877440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870825    2575 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 11:34:19.877440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870828    2575 flags.go:64] FLAG: --help="false"
Sep 11 11:34:19.877440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870832    2575 flags.go:64] FLAG: --hostname-override="ip-10-0-135-16.ec2.internal"
Sep 11 11:34:19.877440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870835    2575 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 11:34:19.877440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870837    2575 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 11:34:19.877440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870840    2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 11:34:19.877440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870843    2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 11:34:19.877440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870846    2575 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 11:34:19.877440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870849    2575 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 11:34:19.877440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870852    2575 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 11:34:19.877440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870855    2575 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 11:34:19.877440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870857    2575 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 11:34:19.877440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870860    2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870863    2575 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870868    2575 flags.go:64] FLAG: --kube-reserved=""
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870871    2575 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870874    2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870877    2575 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870879    2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870882    2575 flags.go:64] FLAG: --lock-file=""
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870884    2575 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870888    2575 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870890    2575 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870896    2575 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870898    2575 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870901    2575 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870904    2575 flags.go:64] FLAG: --logging-format="text"
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870907    2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870910    2575 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870912    2575 flags.go:64] FLAG: --manifest-url=""
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870915    2575 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870919    2575 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870922    2575 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870926    2575 flags.go:64] FLAG: --max-pods="110"
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870929    2575 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870932    2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870935    2575 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 11:34:19.878085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870938    2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870941    2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870944    2575 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870947    2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870954    2575 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870957    2575 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870960    2575 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870963    2575 flags.go:64] FLAG: --pod-cidr=""
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870970    2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870975    2575 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870979    2575 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870982    2575 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870985    2575 flags.go:64] FLAG: --port="10250"
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870988    2575 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870991    2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04832abd704b66904"
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870994    2575 flags.go:64] FLAG: --qos-reserved=""
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.870997    2575 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871000    2575 flags.go:64] FLAG: --register-node="true"
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871002    2575 flags.go:64] FLAG: --register-with-taints=""
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871006    2575 flags.go:64] FLAG: --registry-burst="10"
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871009    2575 flags.go:64] FLAG: --registry-qps="5"
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871012    2575 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871014    2575 flags.go:64] FLAG: --reserved-memory=""
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871018    2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871021    2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 11:34:19.878732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871024    2575 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871027    2575 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871030    2575 flags.go:64] FLAG: --runonce="false"
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871032    2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871036    2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871039    2575 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871042    2575 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871044    2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871047    2575 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871050    2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871053    2575 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871056    2575 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871059    2575 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871062    2575 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871064    2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871067    2575 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871071    2575 flags.go:64] FLAG: --system-cgroups=""
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871075    2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871080    2575 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871083    2575 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871085    2575 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871090    2575 flags.go:64] FLAG: --tls-min-version=""
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871092    2575 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871095    2575 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871098    2575 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 11:34:19.879352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871101    2575 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 11:34:19.879972 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871103    2575 flags.go:64] FLAG: --v="2"
Sep 11 11:34:19.879972 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871107    2575 flags.go:64] FLAG: --version="false"
Sep 11 11:34:19.879972 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871111    2575 flags.go:64] FLAG: --vmodule=""
Sep 11 11:34:19.879972 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871115    2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 11:34:19.879972 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871118    2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 11:34:19.879972 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871244    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:34:19.879972 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871248    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:34:19.879972 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871251    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:34:19.879972 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871254    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:34:19.879972 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871257    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:34:19.879972 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871261    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:34:19.879972 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871263    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:34:19.879972 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871267    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:34:19.879972 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871271    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:34:19.879972 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871273    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:34:19.879972 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871277    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:34:19.879972 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871280    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:34:19.879972 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871282    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:34:19.879972 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871285    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:34:19.879972 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871287    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:34:19.879972 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871298    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:34:19.880483 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871302    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:34:19.880483 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871305    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:34:19.880483 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871309    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:34:19.880483 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871312    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:34:19.880483 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871315    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:34:19.880483 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871317    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:34:19.880483 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871320    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:34:19.880483 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871322    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:34:19.880483 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871325    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:34:19.880483 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871327    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:34:19.880483 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871330    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:34:19.880483 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871333    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:34:19.880483 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871335    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:34:19.880483 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871338    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:34:19.880483 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871341    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:34:19.880483 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871343    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:34:19.880483 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871346    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:34:19.880483 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871349    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:34:19.880483 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871351    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:34:19.880483 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871354    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:34:19.881329 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871356    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:34:19.881329 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871360    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:34:19.881329 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871363    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:34:19.881329 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871365    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:34:19.881329 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871368    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:34:19.881329 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871371    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:34:19.881329 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871373    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:34:19.881329 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871376    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:34:19.881329 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871378    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:34:19.881329 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871381    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:34:19.881329 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871383    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:34:19.881329 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871386    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:34:19.881329 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871388    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:34:19.881329 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871391    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:34:19.881329 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871394    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:34:19.881329 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871397    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:34:19.881329 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871400    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:34:19.881329 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871403    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:34:19.881329 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871405    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:34:19.881954 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871408    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:34:19.881954 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871410    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:34:19.881954 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871413    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:34:19.881954 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871416    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:34:19.881954 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871418    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:34:19.881954 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871420    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:34:19.881954 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871423    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:34:19.881954 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871426    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:34:19.881954 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871428    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:34:19.881954 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871431    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:34:19.881954 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871434    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:34:19.881954 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871436    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:34:19.881954 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871438    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:34:19.881954 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871441    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:34:19.881954 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871445    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:34:19.881954 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871447    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:34:19.881954 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871450    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:34:19.881954 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871452    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:34:19.881954 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871455    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:34:19.881954 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871458    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:34:19.882440 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871460    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:34:19.882440 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871463    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:34:19.882440 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871465    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:34:19.882440 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871469    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:34:19.882440 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871471    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:34:19.882440 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871474    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:34:19.882440 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871476    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:34:19.882440 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871480    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:34:19.882440 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871483    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:34:19.882440 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871486    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:34:19.882440 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871489    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:34:19.882440 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871493    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:34:19.882440 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871496    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:34:19.882440 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871498    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:34:19.882440 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871501    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:34:19.882440 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871504    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:34:19.882440 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871507    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:34:19.882440 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871509    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:34:19.882440 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871513    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:34:19.882931 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871518    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:34:19.882931 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871521    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:34:19.882931 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871523    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:34:19.882931 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871526    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:34:19.882931 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.871528    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:34:19.882931 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.871533    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 11:34:19.882931 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.878743    2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 11:34:19.882931 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.878759    2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 11:34:19.882931 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.879517    2575 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 11:34:19.882931 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.879525    2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 11:34:19.882931 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879593    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:34:19.882931 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879599    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:34:19.882931 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879602    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:34:19.882931 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879605    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:34:19.882931 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879608    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:34:19.883334 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879611    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:34:19.883334 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879614    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:34:19.883334 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879616    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:34:19.883334 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879619    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:34:19.883334 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879622    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:34:19.883334 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879624    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:34:19.883334 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879627    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:34:19.883334 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879645    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:34:19.883334 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879648    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:34:19.883334 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879651    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:34:19.883334 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879653    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:34:19.883334 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879656    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:34:19.883334 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879659    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:34:19.883334 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879662    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:34:19.883334 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879665    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:34:19.883334 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879668    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:34:19.883334 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879671    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:34:19.883334 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879673    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:34:19.883334 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879677    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:34:19.883334 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879679    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:34:19.883848 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879682    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:34:19.883848 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879685    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:34:19.883848 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879688    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:34:19.883848 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879692    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:34:19.883848 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879699    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:34:19.883848 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879702    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:34:19.883848 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879705    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:34:19.883848 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879708    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:34:19.883848 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879710    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:34:19.883848 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879713    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:34:19.883848 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879717    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:34:19.883848 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879721    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:34:19.883848 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879724    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:34:19.883848 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879727    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:34:19.883848 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879730    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:34:19.883848 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879733    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:34:19.883848 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879736    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:34:19.883848 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879739    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:34:19.883848 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879742    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:34:19.884324 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879744    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:34:19.884324 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879747    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:34:19.884324 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879749    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:34:19.884324 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879752    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:34:19.884324 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879755    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:34:19.884324 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879758    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:34:19.884324 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879760    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:34:19.884324 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879763    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:34:19.884324 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879765    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:34:19.884324 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879768    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:34:19.884324 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879770    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:34:19.884324 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879773    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:34:19.884324 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879775    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:34:19.884324 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879778    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:34:19.884324 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879781    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:34:19.884324 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879784    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:34:19.884324 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879786    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:34:19.884324 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879789    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:34:19.884324 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879792    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:34:19.884324 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879795    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:34:19.884833 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879798    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:34:19.884833 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879800    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:34:19.884833 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879802    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:34:19.884833 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879805    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:34:19.884833 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879808    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:34:19.884833 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879810    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:34:19.884833 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879813    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:34:19.884833 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879815    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:34:19.884833 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879818    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:34:19.884833 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879820    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:34:19.884833 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879823    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:34:19.884833 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879826    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:34:19.884833 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879829    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:34:19.884833 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879831    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:34:19.884833 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879835    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:34:19.884833 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879838    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:34:19.884833 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879842    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:34:19.884833 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879845    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:34:19.884833 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879847    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:34:19.885348 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879850    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:34:19.885348 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879853    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:34:19.885348 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879856    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:34:19.885348 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879859    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:34:19.885348 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879861    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:34:19.885348 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879864    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:34:19.885348 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879867    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:34:19.885348 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879869    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:34:19.885348 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879872    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:34:19.885348 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879874    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:34:19.885348 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879877    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:34:19.885348 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879880    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:34:19.885348 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879883    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:34:19.885348 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879885    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:34:19.885348 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879888    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:34:19.885348 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.879891    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:34:19.885839 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.879896    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 11:34:19.885839 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880027    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 11:34:19.885839 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880032    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 11:34:19.885839 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880036    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 11:34:19.885839 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880040    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 11:34:19.885839 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880043    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 11:34:19.885839 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880047    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 11:34:19.885839 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880050    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 11:34:19.885839 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880053    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 11:34:19.885839 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880056    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 11:34:19.885839 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880059    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 11:34:19.885839 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880062    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 11:34:19.885839 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880065    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 11:34:19.885839 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880068    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 11:34:19.886189 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880071    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 11:34:19.886189 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880073    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 11:34:19.886189 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880076    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 11:34:19.886189 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880079    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 11:34:19.886189 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880082    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 11:34:19.886189 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880084    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 11:34:19.886189 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880086    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 11:34:19.886189 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880089    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 11:34:19.886189 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880092    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 11:34:19.886189 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880094    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 11:34:19.886189 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880097    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 11:34:19.886189 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880099    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 11:34:19.886189 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880102    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 11:34:19.886189 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880105    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 11:34:19.886189 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880107    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 11:34:19.886189 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880110    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 11:34:19.886189 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880112    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 11:34:19.886189 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880115    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 11:34:19.886189 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880117    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 11:34:19.886189 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880120    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 11:34:19.886704 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880122    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 11:34:19.886704 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880125    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 11:34:19.886704 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880127    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 11:34:19.886704 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880129    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 11:34:19.886704 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880132    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 11:34:19.886704 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880134    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 11:34:19.886704 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880137    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 11:34:19.886704 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880139    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 11:34:19.886704 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880142    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 11:34:19.886704 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880144    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 11:34:19.886704 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880147    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 11:34:19.886704 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880149    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 11:34:19.886704 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880152    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 11:34:19.886704 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880154    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 11:34:19.886704 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880157    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 11:34:19.886704 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880160    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 11:34:19.886704 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880162    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 11:34:19.886704 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880165    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 11:34:19.886704 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880167    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 11:34:19.886704 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880170    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 11:34:19.887205 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880172    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 11:34:19.887205 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880175    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 11:34:19.887205 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880177    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 11:34:19.887205 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880180    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 11:34:19.887205 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880183    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 11:34:19.887205 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880185    2575 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 11:34:19.887205 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880188    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 11:34:19.887205 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880190    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 11:34:19.887205 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880193    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 11:34:19.887205 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880196    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 11:34:19.887205 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880198    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 11:34:19.887205 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880200    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 11:34:19.887205 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880203    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 11:34:19.887205 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880205    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 11:34:19.887205 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880208    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 11:34:19.887205 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880210    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 11:34:19.887205 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880213    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 11:34:19.887205 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880215    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 11:34:19.887205 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880218    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 11:34:19.887205 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880220    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 11:34:19.887707 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880223    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 11:34:19.887707 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880225    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 11:34:19.887707 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880228    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 11:34:19.887707 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880230    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 11:34:19.887707 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880233    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 11:34:19.887707 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880236    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 11:34:19.887707 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880238    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 11:34:19.887707 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880241    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 11:34:19.887707 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880244    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 11:34:19.887707 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880247    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 11:34:19.887707 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880249    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 11:34:19.887707 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880252    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 11:34:19.887707 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880255    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 11:34:19.887707 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880259    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 11:34:19.887707 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880261    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 11:34:19.887707 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880264    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 11:34:19.887707 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880266    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 11:34:19.887707 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880269    2575 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 11:34:19.887707 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880271    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 11:34:19.887707 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880274    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 11:34:19.888237 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880277    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 11:34:19.888237 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880279    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 11:34:19.888237 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880282    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 11:34:19.888237 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880284    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 11:34:19.888237 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880287    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 11:34:19.888237 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:19.880289    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 11:34:19.888237 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.880294    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 11:34:19.888237 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.880430    2575 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 11:34:19.888237 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.883378    2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 11:34:19.888237 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.884260    2575 server.go:1019] "Starting client certificate rotation"
Sep 11 11:34:19.888237 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.884359    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 11:34:19.888237 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.884396    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 11:34:19.888237 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.886916    2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 11:34:19.888578 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.888035    2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 11:34:19.897449 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.897432    2575 log.go:25] "Validated CRI v1 runtime API"
Sep 11 11:34:19.900209 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.900194    2575 log.go:25] "Validated CRI v1 image API"
Sep 11 11:34:19.901054 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.901036    2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 11:34:19.904840 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.904816    2575 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9ac2c7f1-0321-4eaa-a69c-97d6063a410b:/dev/nvme0n1p3 a09e963d-3ac3-48bb-852f-3967db73f9e1:/dev/nvme0n1p4]
Sep 11 11:34:19.904895 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.904840    2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 11:34:19.910819 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.910713    2575 manager.go:216] Machine: {Timestamp:2026-09-11 11:34:19.90934485 +0000 UTC m=+0.329950869 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103170 MemoryCapacity:32812154880 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24532d432229dbd4ad56190d93a362 SystemUUID:ec24532d-4322-29db-d4ad-56190d93a362 BootID:389e4846-59c3-454a-b040-2bb28b82b9d2 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2b:f6:40:40:3f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2b:f6:40:40:3f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:45:fd:7c:82:da Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812154880 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 11:34:19.910819 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.910810    2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 11:34:19.910935 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.910885    2575 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 11:34:19.913693 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.913664    2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 11:34:19.913825 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.913696    2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-16.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 11:34:19.914159 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.914143    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 11:34:19.914504 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.914492    2575 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 11:34:19.914556 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.914506    2575 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 11:34:19.914556 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.914514    2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 11:34:19.915100 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.915089    2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 11:34:19.915928 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.915918    2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 11:34:19.920507 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.920496    2575 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 11:34:19.920613 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.920604    2575 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 11:34:19.920667 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.920656    2575 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 11:34:19.920667 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.920666    2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 11:34:19.920741 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.920679    2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 11:34:19.920741 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.920688    2575 kubelet.go:398] "Adding apiserver pod source"
Sep 11 11:34:19.920741 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.920696    2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 11:34:19.921877 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.921852    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 11:34:19.921966 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.921893    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 11:34:19.924397 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.924379    2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 11:34:19.926116 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.926103    2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 11:34:19.926158 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.926133    2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 11:34:19.927848 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.927836    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 11:34:19.927901 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.927853    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 11:34:19.927901 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.927859    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 11:34:19.927901 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.927866    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 11:34:19.927901 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.927878    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 11:34:19.927901 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.927884    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 11:34:19.927901 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.927902    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 11:34:19.928058 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.927907    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 11:34:19.928058 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.927914    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 11:34:19.928058 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.927920    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 11:34:19.928058 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.927929    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 11:34:19.928058 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.927934    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 11:34:19.928611 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.928603    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 11:34:19.928611 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.928611    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 11:34:19.932319 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.932305    2575 server.go:1268] "Started kubelet"
Sep 11 11:34:19.932436 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.932379    2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 11:34:19.932574 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:19.932548    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 11:34:19.932694 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.932621    2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 11:34:19.932747 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.932700    2575 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 11:34:19.932747 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.932708    2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-16.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 11:34:19.932858 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:19.932842    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-16.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 11:34:19.933239 ip-10-0-135-16 systemd[1]: Started Kubernetes Kubelet.
Sep 11 11:34:19.934665 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.934621    2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 11:34:19.935054 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.935040    2575 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 11:34:19.936134 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.936117    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8m6rk"
Sep 11 11:34:19.936479 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:19.936464    2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 11:34:19.937924 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:19.936952    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-16.ec2.internal.18d441093e8401fb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-16.ec2.internal,UID:ip-10-0-135-16.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-16.ec2.internal,},FirstTimestamp:2026-09-11 11:34:19.932279291 +0000 UTC m=+0.352885308,LastTimestamp:2026-09-11 11:34:19.932279291 +0000 UTC m=+0.352885308,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-16.ec2.internal,}"
Sep 11 11:34:19.938696 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.938682    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 11:34:19.939374 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.939358    2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 11:34:19.939462 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:19.939432    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-16.ec2.internal\" not found"
Sep 11 11:34:19.939930 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.939910    2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 11:34:19.940063 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.940049    2575 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 11:34:19.940116 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.940066    2575 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 11:34:19.940178 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.940125    2575 factory.go:55] Registering systemd factory
Sep 11 11:34:19.940178 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.940137    2575 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 11:34:19.940178 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.940155    2575 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 11:34:19.940178 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.940162    2575 factory.go:223] Registration of the systemd container factory successfully
Sep 11 11:34:19.940389 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.940376    2575 factory.go:153] Registering CRI-O factory
Sep 11 11:34:19.940389 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.940388    2575 factory.go:223] Registration of the crio container factory successfully
Sep 11 11:34:19.940489 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.940425    2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 11:34:19.940489 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.940437    2575 factory.go:103] Registering Raw factory
Sep 11 11:34:19.940489 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.940449    2575 manager.go:1196] Started watching for new ooms in manager
Sep 11 11:34:19.940824 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.940812    2575 manager.go:349] Starting recovery of all containers
Sep 11 11:34:19.943853 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.943826    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8m6rk"
Sep 11 11:34:19.950295 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:19.950265    2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-16.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 11:34:19.950485 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:19.950445    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 11:34:19.953793 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.953780    2575 manager.go:354] Recovery completed
Sep 11 11:34:19.957708 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.957695    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:34:19.959726 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.959712    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:34:19.959792 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.959738    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:34:19.959792 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.959749    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:34:19.960347 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.960335    2575 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 11:34:19.960347 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.960345    2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 11:34:19.960447 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.960360    2575 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 11:34:19.960447 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.960385    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:34:19.961179 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.961160    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:34:19.961263 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.961189    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:34:19.961263 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.961199    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:34:19.962436 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.962422    2575 policy_none.go:49] "None policy: Start"
Sep 11 11:34:19.962436 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.962439    2575 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 11:34:19.962516 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.962448    2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 11:34:19.964751 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:19.964736    2575 policy_none.go:47] "Start"
Sep 11 11:34:20.021926 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.002007    2575 manager.go:337] "Starting Device Plugin manager"
Sep 11 11:34:20.021926 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:20.002163    2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 11:34:20.021926 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.002175    2575 server.go:87] "Starting device plugin registration server"
Sep 11 11:34:20.021926 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.002375    2575 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 11:34:20.021926 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.002386    2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 11:34:20.021926 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.002482    2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 11:34:20.021926 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.002556    2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 11:34:20.021926 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.002564    2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 11:34:20.021926 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:20.003134    2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 11:34:20.021926 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:20.003179    2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-16.ec2.internal\" not found"
Sep 11 11:34:20.073510 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.073480    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 11:34:20.074787 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.074770    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 11:34:20.074851 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.074821    2575 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 11:34:20.074851 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.074845    2575 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 11:34:20.074941 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:20.074923    2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 11:34:20.077191 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.077171    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 11:34:20.103165 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.103141    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:34:20.104007 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.103984    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:34:20.104101 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.104015    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:34:20.104101 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.104026    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:34:20.104101 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.104046    2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-16.ec2.internal"
Sep 11 11:34:20.112047 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.112031    2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-16.ec2.internal"
Sep 11 11:34:20.112135 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:20.112056    2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-16.ec2.internal\": node \"ip-10-0-135-16.ec2.internal\" not found"
Sep 11 11:34:20.137323 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:20.137275    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-16.ec2.internal\" not found"
Sep 11 11:34:20.175576 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.175546    2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-16.ec2.internal"]
Sep 11 11:34:20.175668 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.175618    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:34:20.176423 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.176408    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:34:20.176491 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.176437    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:34:20.176491 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.176447    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:34:20.178741 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.178727    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:34:20.178860 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.178845    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal"
Sep 11 11:34:20.178909 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.178877    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:34:20.179436 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.179420    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:34:20.179505 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.179448    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:34:20.179505 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.179458    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:34:20.179860 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.179841    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:34:20.179925 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.179870    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:34:20.179925 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.179887    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:34:20.182121 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.182106    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-16.ec2.internal"
Sep 11 11:34:20.182188 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.182129    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 11:34:20.182815 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.182800    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 11:34:20.182889 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.182826    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 11:34:20.182889 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.182837    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeHasSufficientPID"
Sep 11 11:34:20.202405 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:20.202386    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-16.ec2.internal\" not found" node="ip-10-0-135-16.ec2.internal"
Sep 11 11:34:20.206762 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:20.206745    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-16.ec2.internal\" not found" node="ip-10-0-135-16.ec2.internal"
Sep 11 11:34:20.238001 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:20.237984    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-16.ec2.internal\" not found"
Sep 11 11:34:20.242338 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.242320    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e055cf25dcc047127c65d7598806ffba-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal\" (UID: \"e055cf25dcc047127c65d7598806ffba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal"
Sep 11 11:34:20.242438 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.242343    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e055cf25dcc047127c65d7598806ffba-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal\" (UID: \"e055cf25dcc047127c65d7598806ffba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal"
Sep 11 11:34:20.242438 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.242360    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a7293eba94400e360077848fce69243b-config\") pod \"kube-apiserver-proxy-ip-10-0-135-16.ec2.internal\" (UID: \"a7293eba94400e360077848fce69243b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-16.ec2.internal"
Sep 11 11:34:20.339068 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:20.339048    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-16.ec2.internal\" not found"
Sep 11 11:34:20.343339 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.343324    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e055cf25dcc047127c65d7598806ffba-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal\" (UID: \"e055cf25dcc047127c65d7598806ffba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal"
Sep 11 11:34:20.343390 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.343349    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a7293eba94400e360077848fce69243b-config\") pod \"kube-apiserver-proxy-ip-10-0-135-16.ec2.internal\" (UID: \"a7293eba94400e360077848fce69243b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-16.ec2.internal"
Sep 11 11:34:20.343390 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.343376    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e055cf25dcc047127c65d7598806ffba-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal\" (UID: \"e055cf25dcc047127c65d7598806ffba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal"
Sep 11 11:34:20.343468 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.343411    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e055cf25dcc047127c65d7598806ffba-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal\" (UID: \"e055cf25dcc047127c65d7598806ffba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal"
Sep 11 11:34:20.343468 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.343418    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e055cf25dcc047127c65d7598806ffba-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal\" (UID: \"e055cf25dcc047127c65d7598806ffba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal"
Sep 11 11:34:20.343468 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.343426    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a7293eba94400e360077848fce69243b-config\") pod \"kube-apiserver-proxy-ip-10-0-135-16.ec2.internal\" (UID: \"a7293eba94400e360077848fce69243b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-16.ec2.internal"
Sep 11 11:34:20.440176 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:20.440100    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-16.ec2.internal\" not found"
Sep 11 11:34:20.504418 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.504392    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal"
Sep 11 11:34:20.510074 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.510057    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-16.ec2.internal"
Sep 11 11:34:20.540658 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:20.540623    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-16.ec2.internal\" not found"
Sep 11 11:34:20.641119 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:20.641086    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-16.ec2.internal\" not found"
Sep 11 11:34:20.741526 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:20.741479    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-16.ec2.internal\" not found"
Sep 11 11:34:20.841963 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:20.841943    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-16.ec2.internal\" not found"
Sep 11 11:34:20.884234 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.884217    2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 11:34:20.884713 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.884329    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 11:34:20.942860 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:20.942843    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-16.ec2.internal\" not found"
Sep 11 11:34:20.949565 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.949538    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 11:29:19 +0000 UTC" deadline="2028-06-01 02:55:13.439769442 +0000 UTC"
Sep 11 11:34:20.949565 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:20.949563    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15087h20m52.490209582s"
Sep 11 11:34:21.043274 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:21.043206    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-16.ec2.internal\" not found"
Sep 11 11:34:21.092783 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:21.092746    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7293eba94400e360077848fce69243b.slice/crio-e308829b9ffa790431c666bfe16422fb2f18f20abab5f83f107b77d4418f24e1 WatchSource:0}: Error finding container e308829b9ffa790431c666bfe16422fb2f18f20abab5f83f107b77d4418f24e1: Status 404 returned error can't find the container with id e308829b9ffa790431c666bfe16422fb2f18f20abab5f83f107b77d4418f24e1
Sep 11 11:34:21.093056 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:21.093031    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode055cf25dcc047127c65d7598806ffba.slice/crio-1c16bb768c4c9249a534f57606d8a468ab11eb755c241344f092195334294efc WatchSource:0}: Error finding container 1c16bb768c4c9249a534f57606d8a468ab11eb755c241344f092195334294efc: Status 404 returned error can't find the container with id 1c16bb768c4c9249a534f57606d8a468ab11eb755c241344f092195334294efc
Sep 11 11:34:21.097746 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.097731    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 11:34:21.144158 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:21.144124    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-16.ec2.internal\" not found"
Sep 11 11:34:21.233998 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.233975    2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 11:34:21.245189 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:21.245168    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-16.ec2.internal\" not found"
Sep 11 11:34:21.282791 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.282771    2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 11:34:21.323511 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.323466    2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 11:34:21.340392 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.340363    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-16.ec2.internal"
Sep 11 11:34:21.349848 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.349830    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 11:34:21.350874 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.350858    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal"
Sep 11 11:34:21.361079 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.361062    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 11:34:21.921922 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.921849    2575 apiserver.go:52] "Watching apiserver"
Sep 11 11:34:21.928844 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.928821    2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 11:34:21.930764 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.930736    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc","openshift-cluster-node-tuning-operator/tuned-x7p9s","openshift-dns/node-resolver-vf4g9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal","openshift-multus/multus-tc2q8","openshift-network-operator/iptables-alerter-679fd","kube-system/global-pull-secret-syncer-nhrgs","openshift-image-registry/node-ca-lx4vv","openshift-multus/multus-additional-cni-plugins-flt4d","openshift-multus/network-metrics-daemon-2tbn7","openshift-network-diagnostics/network-check-target-wklxl","openshift-ovn-kubernetes/ovnkube-node-rzhmb","kube-system/konnectivity-agent-6tqm4","kube-system/kube-apiserver-proxy-ip-10-0-135-16.ec2.internal"]
Sep 11 11:34:21.933007 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.932986    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lx4vv"
Sep 11 11:34:21.934957 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.934931    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:21.935052 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:21.934999    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:34:21.935172 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.935157    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 11:34:21.935264 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.935247    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 11:34:21.935315 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.935272    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-j2psn\""
Sep 11 11:34:21.935315 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.935246    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 11:34:21.936960 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.936941    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:21.937050 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:21.937001    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:34:21.938893 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.938879    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:21.939419 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.939404    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 11:34:21.940951 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.940937    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-679fd"
Sep 11 11:34:21.941456 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.941438    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-l5hn4\""
Sep 11 11:34:21.941537 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.941487    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 11:34:21.941537 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.941498    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 11:34:21.941714 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.941693    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 11:34:21.941799 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.941694    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 11:34:21.942967 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.942952    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:21.943053 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.942976    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 11:34:21.943053 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:21.943002    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:34:21.943332 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.943315    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 11:34:21.943765 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.943561    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zvf8j\""
Sep 11 11:34:21.943861 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.943843    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 11:34:21.945195 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.945178    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vf4g9"
Sep 11 11:34:21.947155 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.947141    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-flt4d"
Sep 11 11:34:21.947437 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.947423    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 11:34:21.948294 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.948277    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pxl55\""
Sep 11 11:34:21.948573 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.948558    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 11:34:21.949153 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.949135    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 11:34:21.949414 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.949401    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc"
Sep 11 11:34:21.951035 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.951012    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ee3a4a6-522e-4174-a9aa-5da4e429d371-cni-binary-copy\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:21.951144 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.951065    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-host-var-lib-cni-multus\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:21.951144 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.951097    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-hostroot\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:21.951144 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.951119    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-multus-conf-dir\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:21.951270 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.951146    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-host-run-multus-certs\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:21.951270 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.951171    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-etc-kubernetes\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:21.951270 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.951197    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9470c69c-fd20-49fc-8b0f-e02d3010e1bc-host\") pod \"node-ca-lx4vv\" (UID: \"9470c69c-fd20-49fc-8b0f-e02d3010e1bc\") " pod="openshift-image-registry/node-ca-lx4vv"
Sep 11 11:34:21.951270 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.951218    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs\") pod \"network-metrics-daemon-2tbn7\" (UID: \"7d81bb62-6b31-4915-8827-f8d7347665c0\") " pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:21.951270 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.951247    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-multus-cni-dir\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:21.951483 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.951282    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-cnibin\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:21.951483 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.951308    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-host-run-k8s-cni-cncf-io\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:21.951483 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.951335    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8608f214-d97e-496d-936e-fdc852376d73-iptables-alerter-script\") pod \"iptables-alerter-679fd\" (UID: \"8608f214-d97e-496d-936e-fdc852376d73\") " pod="openshift-network-operator/iptables-alerter-679fd"
Sep 11 11:34:21.951483 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.951378    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8nkh\" (UniqueName: \"kubernetes.io/projected/9470c69c-fd20-49fc-8b0f-e02d3010e1bc-kube-api-access-d8nkh\") pod \"node-ca-lx4vv\" (UID: \"9470c69c-fd20-49fc-8b0f-e02d3010e1bc\") " pod="openshift-image-registry/node-ca-lx4vv"
Sep 11 11:34:21.951483 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.951417    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmxn7\" (UniqueName: \"kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7\") pod \"network-check-target-wklxl\" (UID: \"0b9f399e-7383-4c4a-897c-fa71548f24d1\") " pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:21.951745 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.951728    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-np7v8\""
Sep 11 11:34:21.951925 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.951462    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-host-run-netns\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:21.952000 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.951945    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-host-var-lib-cni-bin\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:21.952000 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.951967    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-host-var-lib-kubelet\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:21.952000 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.951992    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0ee3a4a6-522e-4174-a9aa-5da4e429d371-multus-daemon-config\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:21.952142 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.952017    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p28nq\" (UniqueName: \"kubernetes.io/projected/8608f214-d97e-496d-936e-fdc852376d73-kube-api-access-p28nq\") pod \"iptables-alerter-679fd\" (UID: \"8608f214-d97e-496d-936e-fdc852376d73\") " pod="openshift-network-operator/iptables-alerter-679fd"
Sep 11 11:34:21.952142 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.952043    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9470c69c-fd20-49fc-8b0f-e02d3010e1bc-serviceca\") pod \"node-ca-lx4vv\" (UID: \"9470c69c-fd20-49fc-8b0f-e02d3010e1bc\") " pod="openshift-image-registry/node-ca-lx4vv"
Sep 11 11:34:21.952142 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.952066    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq87p\" (UniqueName: \"kubernetes.io/projected/7d81bb62-6b31-4915-8827-f8d7347665c0-kube-api-access-nq87p\") pod \"network-metrics-daemon-2tbn7\" (UID: \"7d81bb62-6b31-4915-8827-f8d7347665c0\") " pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:21.952142 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.952107    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-system-cni-dir\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:21.952142 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.952116    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 11:34:21.952142 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.952132    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-multus-socket-dir-parent\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:21.952400 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.952159    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqptb\" (UniqueName: \"kubernetes.io/projected/0ee3a4a6-522e-4174-a9aa-5da4e429d371-kube-api-access-zqptb\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:21.952400 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.952184    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8608f214-d97e-496d-936e-fdc852376d73-host-slash\") pod \"iptables-alerter-679fd\" (UID: \"8608f214-d97e-496d-936e-fdc852376d73\") " pod="openshift-network-operator/iptables-alerter-679fd"
Sep 11 11:34:21.952584 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.952567    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 11:34:21.953159 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.952849    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 11:34:21.953159 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.952871    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-26djp\""
Sep 11 11:34:21.953159 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.952889    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:21.953159 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.952918    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 11:34:21.954910 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.954891    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:21.955402 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.955385    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 11:34:21.955774 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.955756    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 11:34:21.955856 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.955824    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 11:34:21.955924 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.955911    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-f6tzd\""
Sep 11 11:34:21.957153 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.957131    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 11:34:21.957479 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.957465    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6tqm4"
Sep 11 11:34:21.957941 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.957913    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 11:34:21.958719 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.958124    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 11:34:21.958719 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.958158    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-l6k5s\""
Sep 11 11:34:21.958719 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.958220    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 11:34:21.958719 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.958232    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 11:34:21.958719 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.958562    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 11:34:21.959521 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.959502    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fxgb9\""
Sep 11 11:34:21.959605 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.959503    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 11:34:21.959605 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.959543    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 11:34:21.976830 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.976811    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zc9br"
Sep 11 11:34:21.985206 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.985188    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zc9br"
Sep 11 11:34:21.990574 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:21.990556    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 11:34:22.041296 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.041277    2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 11:34:22.052447 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052423    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-log-socket\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.052568 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052456    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/16e07918-992a-4e66-a4c1-e203cc201788-socket-dir\") pod \"aws-ebs-csi-driver-node-djnzc\" (UID: \"16e07918-992a-4e66-a4c1-e203cc201788\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc"
Sep 11 11:34:22.052568 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052478    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-lib-modules\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.052568 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052498    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-etc-openvswitch\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.052568 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052518    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret\") pod \"global-pull-secret-syncer-nhrgs\" (UID: \"e58ed647-be4f-4e08-b5e9-d919687aec38\") " pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:22.052568 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052544    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-etc-kubernetes\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.052568 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052566    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-run-openvswitch\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.052795 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052599    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-host-cni-netd\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.052795 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052645    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs\") pod \"network-metrics-daemon-2tbn7\" (UID: \"7d81bb62-6b31-4915-8827-f8d7347665c0\") " pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:22.052795 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052676    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-multus-cni-dir\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.052795 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052705    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-host-run-k8s-cni-cncf-io\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.052795 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052726    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-etc-sysctl-d\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.052795 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:22.052743    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:34:22.052795 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052746    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-host-run-netns\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.052795 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052780    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/72a9a805-b878-45fd-a7d6-3306117d1975-ovnkube-script-lib\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.052991 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:22.052811    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs podName:7d81bb62-6b31-4915-8827-f8d7347665c0 nodeName:}" failed. No retries permitted until 2026-09-11 11:34:22.552783071 +0000 UTC m=+2.973389088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs") pod "network-metrics-daemon-2tbn7" (UID: "7d81bb62-6b31-4915-8827-f8d7347665c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:34:22.052991 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052842    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9740ecb1-761c-45bc-a8c1-866c71e5c32e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-flt4d\" (UID: \"9740ecb1-761c-45bc-a8c1-866c71e5c32e\") " pod="openshift-multus/multus-additional-cni-plugins-flt4d"
Sep 11 11:34:22.052991 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052849    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-multus-cni-dir\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.052991 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052866    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbl4j\" (UniqueName: \"kubernetes.io/projected/9740ecb1-761c-45bc-a8c1-866c71e5c32e-kube-api-access-pbl4j\") pod \"multus-additional-cni-plugins-flt4d\" (UID: \"9740ecb1-761c-45bc-a8c1-866c71e5c32e\") " pod="openshift-multus/multus-additional-cni-plugins-flt4d"
Sep 11 11:34:22.052991 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052888    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-etc-sysconfig\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.052991 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052892    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-host-run-k8s-cni-cncf-io\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.052991 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052910    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-host-kubelet\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.052991 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052932    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq87p\" (UniqueName: \"kubernetes.io/projected/7d81bb62-6b31-4915-8827-f8d7347665c0-kube-api-access-nq87p\") pod \"network-metrics-daemon-2tbn7\" (UID: \"7d81bb62-6b31-4915-8827-f8d7347665c0\") " pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:22.052991 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052954    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8608f214-d97e-496d-936e-fdc852376d73-host-slash\") pod \"iptables-alerter-679fd\" (UID: \"8608f214-d97e-496d-936e-fdc852376d73\") " pod="openshift-network-operator/iptables-alerter-679fd"
Sep 11 11:34:22.052991 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052975    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9740ecb1-761c-45bc-a8c1-866c71e5c32e-cnibin\") pod \"multus-additional-cni-plugins-flt4d\" (UID: \"9740ecb1-761c-45bc-a8c1-866c71e5c32e\") " pod="openshift-multus/multus-additional-cni-plugins-flt4d"
Sep 11 11:34:22.053254 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.052994    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-etc-modprobe-d\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.053254 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053021    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht6r6\" (UniqueName: \"kubernetes.io/projected/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-kube-api-access-ht6r6\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.053254 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053042    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-host-slash\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.053254 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053063    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8kwz\" (UniqueName: \"kubernetes.io/projected/72a9a805-b878-45fd-a7d6-3306117d1975-kube-api-access-w8kwz\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.053254 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053081    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/412f846c-0794-451d-b9ed-300409e19f26-agent-certs\") pod \"konnectivity-agent-6tqm4\" (UID: \"412f846c-0794-451d-b9ed-300409e19f26\") " pod="kube-system/konnectivity-agent-6tqm4"
Sep 11 11:34:22.053254 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053106    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ee3a4a6-522e-4174-a9aa-5da4e429d371-cni-binary-copy\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.053254 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053125    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-hostroot\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.053254 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053145    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9470c69c-fd20-49fc-8b0f-e02d3010e1bc-host\") pod \"node-ca-lx4vv\" (UID: \"9470c69c-fd20-49fc-8b0f-e02d3010e1bc\") " pod="openshift-image-registry/node-ca-lx4vv"
Sep 11 11:34:22.053254 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053166    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e58ed647-be4f-4e08-b5e9-d919687aec38-kubelet-config\") pod \"global-pull-secret-syncer-nhrgs\" (UID: \"e58ed647-be4f-4e08-b5e9-d919687aec38\") " pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:22.053254 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053188    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-tmp\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.053254 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053209    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/16e07918-992a-4e66-a4c1-e203cc201788-kubelet-dir\") pod \"aws-ebs-csi-driver-node-djnzc\" (UID: \"16e07918-992a-4e66-a4c1-e203cc201788\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc"
Sep 11 11:34:22.053254 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053230    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/16e07918-992a-4e66-a4c1-e203cc201788-registration-dir\") pod \"aws-ebs-csi-driver-node-djnzc\" (UID: \"16e07918-992a-4e66-a4c1-e203cc201788\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc"
Sep 11 11:34:22.053254 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053252    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-cnibin\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.053651 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053273    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8608f214-d97e-496d-936e-fdc852376d73-host-slash\") pod \"iptables-alerter-679fd\" (UID: \"8608f214-d97e-496d-936e-fdc852376d73\") " pod="openshift-network-operator/iptables-alerter-679fd"
Sep 11 11:34:22.053651 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053273    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8nkh\" (UniqueName: \"kubernetes.io/projected/9470c69c-fd20-49fc-8b0f-e02d3010e1bc-kube-api-access-d8nkh\") pod \"node-ca-lx4vv\" (UID: \"9470c69c-fd20-49fc-8b0f-e02d3010e1bc\") " pod="openshift-image-registry/node-ca-lx4vv"
Sep 11 11:34:22.053651 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053310    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmxn7\" (UniqueName: \"kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7\") pod \"network-check-target-wklxl\" (UID: \"0b9f399e-7383-4c4a-897c-fa71548f24d1\") " pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:22.053651 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053341    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9740ecb1-761c-45bc-a8c1-866c71e5c32e-system-cni-dir\") pod \"multus-additional-cni-plugins-flt4d\" (UID: \"9740ecb1-761c-45bc-a8c1-866c71e5c32e\") " pod="openshift-multus/multus-additional-cni-plugins-flt4d"
Sep 11 11:34:22.053651 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053362    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-run\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.053651 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053399    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-host-cni-bin\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.053651 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053421    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/16e07918-992a-4e66-a4c1-e203cc201788-device-dir\") pod \"aws-ebs-csi-driver-node-djnzc\" (UID: \"16e07918-992a-4e66-a4c1-e203cc201788\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc"
Sep 11 11:34:22.053651 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053449    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/16e07918-992a-4e66-a4c1-e203cc201788-etc-selinux\") pod \"aws-ebs-csi-driver-node-djnzc\" (UID: \"16e07918-992a-4e66-a4c1-e203cc201788\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc"
Sep 11 11:34:22.053651 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053471    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-host-run-netns\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.053651 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053481    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-hostroot\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.053651 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053476    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9470c69c-fd20-49fc-8b0f-e02d3010e1bc-host\") pod \"node-ca-lx4vv\" (UID: \"9470c69c-fd20-49fc-8b0f-e02d3010e1bc\") " pod="openshift-image-registry/node-ca-lx4vv"
Sep 11 11:34:22.053651 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053492    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9470c69c-fd20-49fc-8b0f-e02d3010e1bc-serviceca\") pod \"node-ca-lx4vv\" (UID: \"9470c69c-fd20-49fc-8b0f-e02d3010e1bc\") " pod="openshift-image-registry/node-ca-lx4vv"
Sep 11 11:34:22.053651 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053570    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/72a9a805-b878-45fd-a7d6-3306117d1975-ovn-node-metrics-cert\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.053651 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053588    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/16e07918-992a-4e66-a4c1-e203cc201788-sys-fs\") pod \"aws-ebs-csi-driver-node-djnzc\" (UID: \"16e07918-992a-4e66-a4c1-e203cc201788\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc"
Sep 11 11:34:22.053651 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053604    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-system-cni-dir\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.053651 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053642    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-multus-socket-dir-parent\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.054045 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053664    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqptb\" (UniqueName: \"kubernetes.io/projected/0ee3a4a6-522e-4174-a9aa-5da4e429d371-kube-api-access-zqptb\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.054045 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053677    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-etc-systemd\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.054045 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053690    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-systemd-units\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.054045 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053704    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.054045 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053725    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-host-var-lib-cni-multus\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.054045 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053740    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-etc-sysctl-conf\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.054045 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053753    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-host\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.054045 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053804    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8608f214-d97e-496d-936e-fdc852376d73-iptables-alerter-script\") pod \"iptables-alerter-679fd\" (UID: \"8608f214-d97e-496d-936e-fdc852376d73\") " pod="openshift-network-operator/iptables-alerter-679fd"
Sep 11 11:34:22.054045 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053820    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e58ed647-be4f-4e08-b5e9-d919687aec38-dbus\") pod \"global-pull-secret-syncer-nhrgs\" (UID: \"e58ed647-be4f-4e08-b5e9-d919687aec38\") " pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:22.054045 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053834    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9740ecb1-761c-45bc-a8c1-866c71e5c32e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-flt4d\" (UID: \"9740ecb1-761c-45bc-a8c1-866c71e5c32e\") " pod="openshift-multus/multus-additional-cni-plugins-flt4d"
Sep 11 11:34:22.054045 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053848    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-sys\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.054045 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053864    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-var-lib-kubelet\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.054045 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053884    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-etc-tuned\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.054045 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053898    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-var-lib-openvswitch\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.054045 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053911    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-run-ovn\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.054045 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053925    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-host-var-lib-cni-bin\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.054045 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053924    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9470c69c-fd20-49fc-8b0f-e02d3010e1bc-serviceca\") pod \"node-ca-lx4vv\" (UID: \"9470c69c-fd20-49fc-8b0f-e02d3010e1bc\") " pod="openshift-image-registry/node-ca-lx4vv"
Sep 11 11:34:22.054461 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053940    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0ee3a4a6-522e-4174-a9aa-5da4e429d371-multus-daemon-config\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.054461 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053962    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p28nq\" (UniqueName: \"kubernetes.io/projected/8608f214-d97e-496d-936e-fdc852376d73-kube-api-access-p28nq\") pod \"iptables-alerter-679fd\" (UID: \"8608f214-d97e-496d-936e-fdc852376d73\") " pod="openshift-network-operator/iptables-alerter-679fd"
Sep 11 11:34:22.054461 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053989    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-node-log\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.054461 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.053998    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ee3a4a6-522e-4174-a9aa-5da4e429d371-cni-binary-copy\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.054461 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054003    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-host-run-ovn-kubernetes\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.054461 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054035    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-host-var-lib-cni-multus\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.054461 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054042    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/72a9a805-b878-45fd-a7d6-3306117d1975-ovnkube-config\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.054461 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054154    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/72a9a805-b878-45fd-a7d6-3306117d1975-env-overrides\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.054461 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054179    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/412f846c-0794-451d-b9ed-300409e19f26-konnectivity-ca\") pod \"konnectivity-agent-6tqm4\" (UID: \"412f846c-0794-451d-b9ed-300409e19f26\") " pod="kube-system/konnectivity-agent-6tqm4"
Sep 11 11:34:22.054461 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054201    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/88a0717f-18e4-40d9-914c-96ccaaf0e05f-hosts-file\") pod \"node-resolver-vf4g9\" (UID: \"88a0717f-18e4-40d9-914c-96ccaaf0e05f\") " pod="openshift-dns/node-resolver-vf4g9"
Sep 11 11:34:22.054461 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054224    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57jrz\" (UniqueName: \"kubernetes.io/projected/88a0717f-18e4-40d9-914c-96ccaaf0e05f-kube-api-access-57jrz\") pod \"node-resolver-vf4g9\" (UID: \"88a0717f-18e4-40d9-914c-96ccaaf0e05f\") " pod="openshift-dns/node-resolver-vf4g9"
Sep 11 11:34:22.054461 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054248    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8qhl\" (UniqueName: \"kubernetes.io/projected/16e07918-992a-4e66-a4c1-e203cc201788-kube-api-access-t8qhl\") pod \"aws-ebs-csi-driver-node-djnzc\" (UID: \"16e07918-992a-4e66-a4c1-e203cc201788\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc"
Sep 11 11:34:22.054461 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054274    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-multus-conf-dir\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.054461 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054298    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-host-run-multus-certs\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.054461 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054323    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-etc-kubernetes\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.054461 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054345    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88a0717f-18e4-40d9-914c-96ccaaf0e05f-tmp-dir\") pod \"node-resolver-vf4g9\" (UID: \"88a0717f-18e4-40d9-914c-96ccaaf0e05f\") " pod="openshift-dns/node-resolver-vf4g9"
Sep 11 11:34:22.054461 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054368    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9740ecb1-761c-45bc-a8c1-866c71e5c32e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-flt4d\" (UID: \"9740ecb1-761c-45bc-a8c1-866c71e5c32e\") " pod="openshift-multus/multus-additional-cni-plugins-flt4d"
Sep 11 11:34:22.055003 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054393    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-run-systemd\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.055003 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054417    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8608f214-d97e-496d-936e-fdc852376d73-iptables-alerter-script\") pod \"iptables-alerter-679fd\" (UID: \"8608f214-d97e-496d-936e-fdc852376d73\") " pod="openshift-network-operator/iptables-alerter-679fd"
Sep 11 11:34:22.055003 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054419    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-host-var-lib-kubelet\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.055003 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054443    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9740ecb1-761c-45bc-a8c1-866c71e5c32e-cni-binary-copy\") pod \"multus-additional-cni-plugins-flt4d\" (UID: \"9740ecb1-761c-45bc-a8c1-866c71e5c32e\") " pod="openshift-multus/multus-additional-cni-plugins-flt4d"
Sep 11 11:34:22.055003 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054466    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-host-run-netns\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.055003 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054501    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-host-run-multus-certs\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.055003 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054522    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-etc-kubernetes\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.055003 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054528    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0ee3a4a6-522e-4174-a9aa-5da4e429d371-multus-daemon-config\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.055003 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054544    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-multus-conf-dir\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.055003 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054583    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-cnibin\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.055003 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054585    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-host-var-lib-cni-bin\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.055003 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054589    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-host-var-lib-kubelet\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.055003 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054611    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-multus-socket-dir-parent\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.055003 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.054712    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ee3a4a6-522e-4174-a9aa-5da4e429d371-system-cni-dir\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.063277 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.063243    2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 11:34:22.063924 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:22.063903    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:34:22.063924 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:22.063927    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:34:22.064098 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:22.063940    2575 projected.go:196] Error preparing data for projected volume kube-api-access-rmxn7 for pod openshift-network-diagnostics/network-check-target-wklxl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:34:22.064098 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:22.064043    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7 podName:0b9f399e-7383-4c4a-897c-fa71548f24d1 nodeName:}" failed. No retries permitted until 2026-09-11 11:34:22.564010268 +0000 UTC m=+2.984616291 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rmxn7" (UniqueName: "kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7") pod "network-check-target-wklxl" (UID: "0b9f399e-7383-4c4a-897c-fa71548f24d1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:34:22.067339 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.067321    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p28nq\" (UniqueName: \"kubernetes.io/projected/8608f214-d97e-496d-936e-fdc852376d73-kube-api-access-p28nq\") pod \"iptables-alerter-679fd\" (UID: \"8608f214-d97e-496d-936e-fdc852376d73\") " pod="openshift-network-operator/iptables-alerter-679fd"
Sep 11 11:34:22.067440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.067375    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqptb\" (UniqueName: \"kubernetes.io/projected/0ee3a4a6-522e-4174-a9aa-5da4e429d371-kube-api-access-zqptb\") pod \"multus-tc2q8\" (UID: \"0ee3a4a6-522e-4174-a9aa-5da4e429d371\") " pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.067440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.067391    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq87p\" (UniqueName: \"kubernetes.io/projected/7d81bb62-6b31-4915-8827-f8d7347665c0-kube-api-access-nq87p\") pod \"network-metrics-daemon-2tbn7\" (UID: \"7d81bb62-6b31-4915-8827-f8d7347665c0\") " pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:22.067551 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.067441    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8nkh\" (UniqueName: \"kubernetes.io/projected/9470c69c-fd20-49fc-8b0f-e02d3010e1bc-kube-api-access-d8nkh\") pod \"node-ca-lx4vv\" (UID: \"9470c69c-fd20-49fc-8b0f-e02d3010e1bc\") " pod="openshift-image-registry/node-ca-lx4vv"
Sep 11 11:34:22.080734 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.080691    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal" event={"ID":"e055cf25dcc047127c65d7598806ffba","Type":"ContainerStarted","Data":"1c16bb768c4c9249a534f57606d8a468ab11eb755c241344f092195334294efc"}
Sep 11 11:34:22.081956 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.081930    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-16.ec2.internal" event={"ID":"a7293eba94400e360077848fce69243b","Type":"ContainerStarted","Data":"e308829b9ffa790431c666bfe16422fb2f18f20abab5f83f107b77d4418f24e1"}
Sep 11 11:34:22.155575 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.155543    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-node-log\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.155575 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.155574    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-host-run-ovn-kubernetes\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.155786 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.155590    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/72a9a805-b878-45fd-a7d6-3306117d1975-ovnkube-config\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.155786 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.155695    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-node-log\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.155786 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.155744    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/72a9a805-b878-45fd-a7d6-3306117d1975-env-overrides\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.155786 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.155768    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/412f846c-0794-451d-b9ed-300409e19f26-konnectivity-ca\") pod \"konnectivity-agent-6tqm4\" (UID: \"412f846c-0794-451d-b9ed-300409e19f26\") " pod="kube-system/konnectivity-agent-6tqm4"
Sep 11 11:34:22.155786 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.155786    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/88a0717f-18e4-40d9-914c-96ccaaf0e05f-hosts-file\") pod \"node-resolver-vf4g9\" (UID: \"88a0717f-18e4-40d9-914c-96ccaaf0e05f\") " pod="openshift-dns/node-resolver-vf4g9"
Sep 11 11:34:22.155986 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.155801    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57jrz\" (UniqueName: \"kubernetes.io/projected/88a0717f-18e4-40d9-914c-96ccaaf0e05f-kube-api-access-57jrz\") pod \"node-resolver-vf4g9\" (UID: \"88a0717f-18e4-40d9-914c-96ccaaf0e05f\") " pod="openshift-dns/node-resolver-vf4g9"
Sep 11 11:34:22.155986 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.155816    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8qhl\" (UniqueName: \"kubernetes.io/projected/16e07918-992a-4e66-a4c1-e203cc201788-kube-api-access-t8qhl\") pod \"aws-ebs-csi-driver-node-djnzc\" (UID: \"16e07918-992a-4e66-a4c1-e203cc201788\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc"
Sep 11 11:34:22.155986 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.155832    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88a0717f-18e4-40d9-914c-96ccaaf0e05f-tmp-dir\") pod \"node-resolver-vf4g9\" (UID: \"88a0717f-18e4-40d9-914c-96ccaaf0e05f\") " pod="openshift-dns/node-resolver-vf4g9"
Sep 11 11:34:22.155986 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.155851    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9740ecb1-761c-45bc-a8c1-866c71e5c32e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-flt4d\" (UID: \"9740ecb1-761c-45bc-a8c1-866c71e5c32e\") " pod="openshift-multus/multus-additional-cni-plugins-flt4d"
Sep 11 11:34:22.155986 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.155869    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-run-systemd\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.155986 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.155890    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9740ecb1-761c-45bc-a8c1-866c71e5c32e-cni-binary-copy\") pod \"multus-additional-cni-plugins-flt4d\" (UID: \"9740ecb1-761c-45bc-a8c1-866c71e5c32e\") " pod="openshift-multus/multus-additional-cni-plugins-flt4d"
Sep 11 11:34:22.155986 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.155910    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-log-socket\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.155986 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.155930    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/16e07918-992a-4e66-a4c1-e203cc201788-socket-dir\") pod \"aws-ebs-csi-driver-node-djnzc\" (UID: \"16e07918-992a-4e66-a4c1-e203cc201788\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc"
Sep 11 11:34:22.155986 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.155943    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-lib-modules\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.155986 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.155963    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-etc-openvswitch\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.155986 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.155982    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret\") pod \"global-pull-secret-syncer-nhrgs\" (UID: \"e58ed647-be4f-4e08-b5e9-d919687aec38\") " pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:22.156338 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.155996    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-etc-kubernetes\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.156338 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156009    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-run-openvswitch\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.156338 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156021    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-host-cni-netd\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.156338 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156054    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-etc-sysctl-d\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.156338 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156076    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-host-run-netns\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.156338 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156095    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/72a9a805-b878-45fd-a7d6-3306117d1975-ovnkube-script-lib\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.156338 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156109    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9740ecb1-761c-45bc-a8c1-866c71e5c32e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-flt4d\" (UID: \"9740ecb1-761c-45bc-a8c1-866c71e5c32e\") " pod="openshift-multus/multus-additional-cni-plugins-flt4d"
Sep 11 11:34:22.156338 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156122    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbl4j\" (UniqueName: \"kubernetes.io/projected/9740ecb1-761c-45bc-a8c1-866c71e5c32e-kube-api-access-pbl4j\") pod \"multus-additional-cni-plugins-flt4d\" (UID: \"9740ecb1-761c-45bc-a8c1-866c71e5c32e\") " pod="openshift-multus/multus-additional-cni-plugins-flt4d"
Sep 11 11:34:22.156338 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156127    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/72a9a805-b878-45fd-a7d6-3306117d1975-ovnkube-config\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.156338 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156136    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-etc-sysconfig\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.156338 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156155    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-host-kubelet\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.156338 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156171    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9740ecb1-761c-45bc-a8c1-866c71e5c32e-cnibin\") pod \"multus-additional-cni-plugins-flt4d\" (UID: \"9740ecb1-761c-45bc-a8c1-866c71e5c32e\") " pod="openshift-multus/multus-additional-cni-plugins-flt4d"
Sep 11 11:34:22.156338 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156181    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-host-run-ovn-kubernetes\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.156338 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156185    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-etc-modprobe-d\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.156338 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156211    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-etc-openvswitch\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.156338 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156236    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht6r6\" (UniqueName: \"kubernetes.io/projected/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-kube-api-access-ht6r6\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.156338 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156258    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-host-slash\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.157076 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156280    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8kwz\" (UniqueName: \"kubernetes.io/projected/72a9a805-b878-45fd-a7d6-3306117d1975-kube-api-access-w8kwz\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.157076 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156298    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-etc-modprobe-d\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.157076 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156303    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/412f846c-0794-451d-b9ed-300409e19f26-agent-certs\") pod \"konnectivity-agent-6tqm4\" (UID: \"412f846c-0794-451d-b9ed-300409e19f26\") " pod="kube-system/konnectivity-agent-6tqm4"
Sep 11 11:34:22.157076 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156330    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e58ed647-be4f-4e08-b5e9-d919687aec38-kubelet-config\") pod \"global-pull-secret-syncer-nhrgs\" (UID: \"e58ed647-be4f-4e08-b5e9-d919687aec38\") " pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:22.157076 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156344    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-tmp\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.157076 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156360    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/16e07918-992a-4e66-a4c1-e203cc201788-kubelet-dir\") pod \"aws-ebs-csi-driver-node-djnzc\" (UID: \"16e07918-992a-4e66-a4c1-e203cc201788\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc"
Sep 11 11:34:22.157076 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156379    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/16e07918-992a-4e66-a4c1-e203cc201788-registration-dir\") pod \"aws-ebs-csi-driver-node-djnzc\" (UID: \"16e07918-992a-4e66-a4c1-e203cc201788\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc"
Sep 11 11:34:22.157076 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156410    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9740ecb1-761c-45bc-a8c1-866c71e5c32e-system-cni-dir\") pod \"multus-additional-cni-plugins-flt4d\" (UID: \"9740ecb1-761c-45bc-a8c1-866c71e5c32e\") " pod="openshift-multus/multus-additional-cni-plugins-flt4d"
Sep 11 11:34:22.157076 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156422    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-run\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.157076 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156441    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-host-cni-bin\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.157076 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156464    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/16e07918-992a-4e66-a4c1-e203cc201788-device-dir\") pod \"aws-ebs-csi-driver-node-djnzc\" (UID: \"16e07918-992a-4e66-a4c1-e203cc201788\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc"
Sep 11 11:34:22.157076 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156481    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/16e07918-992a-4e66-a4c1-e203cc201788-etc-selinux\") pod \"aws-ebs-csi-driver-node-djnzc\" (UID: \"16e07918-992a-4e66-a4c1-e203cc201788\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc"
Sep 11 11:34:22.157076 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156500    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/72a9a805-b878-45fd-a7d6-3306117d1975-ovn-node-metrics-cert\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.157076 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156515    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/16e07918-992a-4e66-a4c1-e203cc201788-sys-fs\") pod \"aws-ebs-csi-driver-node-djnzc\" (UID: \"16e07918-992a-4e66-a4c1-e203cc201788\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc"
Sep 11 11:34:22.157076 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156538    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-etc-systemd\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.157076 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156560    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-systemd-units\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.157076 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156574    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.157813 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156589    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-etc-sysctl-conf\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.157813 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156612    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-host\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.157813 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156663    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e58ed647-be4f-4e08-b5e9-d919687aec38-dbus\") pod \"global-pull-secret-syncer-nhrgs\" (UID: \"e58ed647-be4f-4e08-b5e9-d919687aec38\") " pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:22.157813 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156687    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9740ecb1-761c-45bc-a8c1-866c71e5c32e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-flt4d\" (UID: \"9740ecb1-761c-45bc-a8c1-866c71e5c32e\") " pod="openshift-multus/multus-additional-cni-plugins-flt4d"
Sep 11 11:34:22.157813 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:22.156752    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:34:22.157813 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:22.156807    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret podName:e58ed647-be4f-4e08-b5e9-d919687aec38 nodeName:}" failed. No retries permitted until 2026-09-11 11:34:22.656790559 +0000 UTC m=+3.077396575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret") pod "global-pull-secret-syncer-nhrgs" (UID: "e58ed647-be4f-4e08-b5e9-d919687aec38") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:34:22.157813 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.156987    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-etc-kubernetes\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.157813 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157035    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-run-openvswitch\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.157813 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157076    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-host-cni-netd\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.157813 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157133    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9740ecb1-761c-45bc-a8c1-866c71e5c32e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-flt4d\" (UID: \"9740ecb1-761c-45bc-a8c1-866c71e5c32e\") " pod="openshift-multus/multus-additional-cni-plugins-flt4d"
Sep 11 11:34:22.157813 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157184    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-etc-sysctl-d\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.157813 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157190    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-etc-sysconfig\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.157813 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157224    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-host-run-netns\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.157813 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157230    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-host-kubelet\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.157813 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157269    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9740ecb1-761c-45bc-a8c1-866c71e5c32e-cnibin\") pod \"multus-additional-cni-plugins-flt4d\" (UID: \"9740ecb1-761c-45bc-a8c1-866c71e5c32e\") " pod="openshift-multus/multus-additional-cni-plugins-flt4d"
Sep 11 11:34:22.157813 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157383    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9740ecb1-761c-45bc-a8c1-866c71e5c32e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-flt4d\" (UID: \"9740ecb1-761c-45bc-a8c1-866c71e5c32e\") " pod="openshift-multus/multus-additional-cni-plugins-flt4d"
Sep 11 11:34:22.157813 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157413    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-sys\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.158669 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157438    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-var-lib-kubelet\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.158669 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157462    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-etc-tuned\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.158669 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157485    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-var-lib-openvswitch\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.158669 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157507    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-run-ovn\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.158669 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157591    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-run-ovn\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.158669 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157652    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/72a9a805-b878-45fd-a7d6-3306117d1975-ovnkube-script-lib\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.158669 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157661    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-sys\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.158669 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157695    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-var-lib-kubelet\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.158669 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157716    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e58ed647-be4f-4e08-b5e9-d919687aec38-kubelet-config\") pod \"global-pull-secret-syncer-nhrgs\" (UID: \"e58ed647-be4f-4e08-b5e9-d919687aec38\") " pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:22.158669 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157748    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/16e07918-992a-4e66-a4c1-e203cc201788-kubelet-dir\") pod \"aws-ebs-csi-driver-node-djnzc\" (UID: \"16e07918-992a-4e66-a4c1-e203cc201788\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc"
Sep 11 11:34:22.158669 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157776    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/16e07918-992a-4e66-a4c1-e203cc201788-registration-dir\") pod \"aws-ebs-csi-driver-node-djnzc\" (UID: \"16e07918-992a-4e66-a4c1-e203cc201788\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc"
Sep 11 11:34:22.158669 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157798    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9740ecb1-761c-45bc-a8c1-866c71e5c32e-system-cni-dir\") pod \"multus-additional-cni-plugins-flt4d\" (UID: \"9740ecb1-761c-45bc-a8c1-866c71e5c32e\") " pod="openshift-multus/multus-additional-cni-plugins-flt4d"
Sep 11 11:34:22.158669 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157823    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-run\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.158669 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157845    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-host-cni-bin\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.158669 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157868    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/16e07918-992a-4e66-a4c1-e203cc201788-device-dir\") pod \"aws-ebs-csi-driver-node-djnzc\" (UID: \"16e07918-992a-4e66-a4c1-e203cc201788\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc"
Sep 11 11:34:22.158669 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.157895    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/16e07918-992a-4e66-a4c1-e203cc201788-etc-selinux\") pod \"aws-ebs-csi-driver-node-djnzc\" (UID: \"16e07918-992a-4e66-a4c1-e203cc201788\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc"
Sep 11 11:34:22.158669 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.158276    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-var-lib-openvswitch\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.159427 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.158361    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-run-systemd\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.159427 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.158744    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/72a9a805-b878-45fd-a7d6-3306117d1975-env-overrides\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.159427 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.158852    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.159427 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.158908    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/16e07918-992a-4e66-a4c1-e203cc201788-sys-fs\") pod \"aws-ebs-csi-driver-node-djnzc\" (UID: \"16e07918-992a-4e66-a4c1-e203cc201788\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc"
Sep 11 11:34:22.159427 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.158954    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-etc-systemd\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.159427 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.158992    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-systemd-units\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.159427 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.159035    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-host\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.159427 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.159150    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-etc-sysctl-conf\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.159427 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.159336    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/88a0717f-18e4-40d9-914c-96ccaaf0e05f-hosts-file\") pod \"node-resolver-vf4g9\" (UID: \"88a0717f-18e4-40d9-914c-96ccaaf0e05f\") " pod="openshift-dns/node-resolver-vf4g9"
Sep 11 11:34:22.160318 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.159669    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/412f846c-0794-451d-b9ed-300409e19f26-konnectivity-ca\") pod \"konnectivity-agent-6tqm4\" (UID: \"412f846c-0794-451d-b9ed-300409e19f26\") " pod="kube-system/konnectivity-agent-6tqm4"
Sep 11 11:34:22.160318 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.159723    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-host-slash\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.160318 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.159761    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-lib-modules\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.160318 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.159942    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9740ecb1-761c-45bc-a8c1-866c71e5c32e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-flt4d\" (UID: \"9740ecb1-761c-45bc-a8c1-866c71e5c32e\") " pod="openshift-multus/multus-additional-cni-plugins-flt4d"
Sep 11 11:34:22.160318 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.159981    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/72a9a805-b878-45fd-a7d6-3306117d1975-log-socket\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.160318 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.160009    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e58ed647-be4f-4e08-b5e9-d919687aec38-dbus\") pod \"global-pull-secret-syncer-nhrgs\" (UID: \"e58ed647-be4f-4e08-b5e9-d919687aec38\") " pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:22.160318 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.160101    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/16e07918-992a-4e66-a4c1-e203cc201788-socket-dir\") pod \"aws-ebs-csi-driver-node-djnzc\" (UID: \"16e07918-992a-4e66-a4c1-e203cc201788\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc"
Sep 11 11:34:22.160318 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.160159    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9740ecb1-761c-45bc-a8c1-866c71e5c32e-cni-binary-copy\") pod \"multus-additional-cni-plugins-flt4d\" (UID: \"9740ecb1-761c-45bc-a8c1-866c71e5c32e\") " pod="openshift-multus/multus-additional-cni-plugins-flt4d"
Sep 11 11:34:22.160318 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.160240    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88a0717f-18e4-40d9-914c-96ccaaf0e05f-tmp-dir\") pod \"node-resolver-vf4g9\" (UID: \"88a0717f-18e4-40d9-914c-96ccaaf0e05f\") " pod="openshift-dns/node-resolver-vf4g9"
Sep 11 11:34:22.160714 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.160481    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/72a9a805-b878-45fd-a7d6-3306117d1975-ovn-node-metrics-cert\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.160714 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.160489    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-tmp\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.161000 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.160979    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/412f846c-0794-451d-b9ed-300409e19f26-agent-certs\") pod \"konnectivity-agent-6tqm4\" (UID: \"412f846c-0794-451d-b9ed-300409e19f26\") " pod="kube-system/konnectivity-agent-6tqm4"
Sep 11 11:34:22.161217 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.161193    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-etc-tuned\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.164738 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.164718    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbl4j\" (UniqueName: \"kubernetes.io/projected/9740ecb1-761c-45bc-a8c1-866c71e5c32e-kube-api-access-pbl4j\") pod \"multus-additional-cni-plugins-flt4d\" (UID: \"9740ecb1-761c-45bc-a8c1-866c71e5c32e\") " pod="openshift-multus/multus-additional-cni-plugins-flt4d"
Sep 11 11:34:22.166918 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.166901    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht6r6\" (UniqueName: \"kubernetes.io/projected/313b9fe9-a1e5-4f5a-af78-1bffd2ab7329-kube-api-access-ht6r6\") pod \"tuned-x7p9s\" (UID: \"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329\") " pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.167126 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.167106    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8qhl\" (UniqueName: \"kubernetes.io/projected/16e07918-992a-4e66-a4c1-e203cc201788-kube-api-access-t8qhl\") pod \"aws-ebs-csi-driver-node-djnzc\" (UID: \"16e07918-992a-4e66-a4c1-e203cc201788\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc"
Sep 11 11:34:22.167303 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.167283    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8kwz\" (UniqueName: \"kubernetes.io/projected/72a9a805-b878-45fd-a7d6-3306117d1975-kube-api-access-w8kwz\") pod \"ovnkube-node-rzhmb\" (UID: \"72a9a805-b878-45fd-a7d6-3306117d1975\") " pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.167540 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.167517    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57jrz\" (UniqueName: \"kubernetes.io/projected/88a0717f-18e4-40d9-914c-96ccaaf0e05f-kube-api-access-57jrz\") pod \"node-resolver-vf4g9\" (UID: \"88a0717f-18e4-40d9-914c-96ccaaf0e05f\") " pod="openshift-dns/node-resolver-vf4g9"
Sep 11 11:34:22.244586 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.244517    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lx4vv"
Sep 11 11:34:22.254734 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.254711    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tc2q8"
Sep 11 11:34:22.263341 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.263326    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vf4g9"
Sep 11 11:34:22.269704 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.269688    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-679fd"
Sep 11 11:34:22.276109 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.276088    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-flt4d"
Sep 11 11:34:22.282590 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.282573    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc"
Sep 11 11:34:22.290009 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.289984    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x7p9s"
Sep 11 11:34:22.297409 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.297386    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:22.300777 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.300760    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6tqm4"
Sep 11 11:34:22.560827 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.560751    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs\") pod \"network-metrics-daemon-2tbn7\" (UID: \"7d81bb62-6b31-4915-8827-f8d7347665c0\") " pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:22.560980 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:22.560873    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:34:22.560980 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:22.560935    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs podName:7d81bb62-6b31-4915-8827-f8d7347665c0 nodeName:}" failed. No retries permitted until 2026-09-11 11:34:23.560919114 +0000 UTC m=+3.981525129 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs") pod "network-metrics-daemon-2tbn7" (UID: "7d81bb62-6b31-4915-8827-f8d7347665c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:34:22.661159 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.661130    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmxn7\" (UniqueName: \"kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7\") pod \"network-check-target-wklxl\" (UID: \"0b9f399e-7383-4c4a-897c-fa71548f24d1\") " pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:22.661328 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.661180    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret\") pod \"global-pull-secret-syncer-nhrgs\" (UID: \"e58ed647-be4f-4e08-b5e9-d919687aec38\") " pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:22.661328 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:22.661274    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:34:22.661328 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:22.661289    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:34:22.661328 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:22.661308    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:34:22.661328 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:22.661319    2575 projected.go:196] Error preparing data for projected volume kube-api-access-rmxn7 for pod openshift-network-diagnostics/network-check-target-wklxl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:34:22.661498 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:22.661339    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret podName:e58ed647-be4f-4e08-b5e9-d919687aec38 nodeName:}" failed. No retries permitted until 2026-09-11 11:34:23.66132103 +0000 UTC m=+4.081927035 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret") pod "global-pull-secret-syncer-nhrgs" (UID: "e58ed647-be4f-4e08-b5e9-d919687aec38") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:34:22.661498 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:22.661370    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7 podName:0b9f399e-7383-4c4a-897c-fa71548f24d1 nodeName:}" failed. No retries permitted until 2026-09-11 11:34:23.661355503 +0000 UTC m=+4.081961529 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rmxn7" (UniqueName: "kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7") pod "network-check-target-wklxl" (UID: "0b9f399e-7383-4c4a-897c-fa71548f24d1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:34:22.929889 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:22.929611    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9740ecb1_761c_45bc_a8c1_866c71e5c32e.slice/crio-9ea4c6b5a29d1c18481f49997050c4d43bd7d93857682e54f048d314977963f3 WatchSource:0}: Error finding container 9ea4c6b5a29d1c18481f49997050c4d43bd7d93857682e54f048d314977963f3: Status 404 returned error can't find the container with id 9ea4c6b5a29d1c18481f49997050c4d43bd7d93857682e54f048d314977963f3
Sep 11 11:34:22.930459 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:22.929943    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88a0717f_18e4_40d9_914c_96ccaaf0e05f.slice/crio-15a02e566339c3735e116a4775aefad9674435f31b096936bcbe276a401c5cab WatchSource:0}: Error finding container 15a02e566339c3735e116a4775aefad9674435f31b096936bcbe276a401c5cab: Status 404 returned error can't find the container with id 15a02e566339c3735e116a4775aefad9674435f31b096936bcbe276a401c5cab
Sep 11 11:34:22.932497 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:22.932030    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16e07918_992a_4e66_a4c1_e203cc201788.slice/crio-5a1617c786d50beb108f5ccf09b0c36dd59f869942435d7e895946c722df16ed WatchSource:0}: Error finding container 5a1617c786d50beb108f5ccf09b0c36dd59f869942435d7e895946c722df16ed: Status 404 returned error can't find the container with id 5a1617c786d50beb108f5ccf09b0c36dd59f869942435d7e895946c722df16ed
Sep 11 11:34:22.933047 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:22.932992    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ee3a4a6_522e_4174_a9aa_5da4e429d371.slice/crio-77ea0bc83bf127cd26293683205f79e9ca0a49fdfa3879531f1f537664b1df8d WatchSource:0}: Error finding container 77ea0bc83bf127cd26293683205f79e9ca0a49fdfa3879531f1f537664b1df8d: Status 404 returned error can't find the container with id 77ea0bc83bf127cd26293683205f79e9ca0a49fdfa3879531f1f537664b1df8d
Sep 11 11:34:22.934396 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:22.934291    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72a9a805_b878_45fd_a7d6_3306117d1975.slice/crio-38ff4b5f4b6961e7bdcbd228b528b1db95d97a996bff5377843a9fe4a6c83be7 WatchSource:0}: Error finding container 38ff4b5f4b6961e7bdcbd228b528b1db95d97a996bff5377843a9fe4a6c83be7: Status 404 returned error can't find the container with id 38ff4b5f4b6961e7bdcbd228b528b1db95d97a996bff5377843a9fe4a6c83be7
Sep 11 11:34:22.934884 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:22.934848    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod412f846c_0794_451d_b9ed_300409e19f26.slice/crio-ac37c8620b03fe53d792b00fb02a9efd6c38b98a52bafa2113969dc0079606e9 WatchSource:0}: Error finding container ac37c8620b03fe53d792b00fb02a9efd6c38b98a52bafa2113969dc0079606e9: Status 404 returned error can't find the container with id ac37c8620b03fe53d792b00fb02a9efd6c38b98a52bafa2113969dc0079606e9
Sep 11 11:34:22.953671 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:22.953650    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9470c69c_fd20_49fc_8b0f_e02d3010e1bc.slice/crio-acfffea194d3433763118522b1dcab8ab92764d16452ae9333cffbb600fbbf65 WatchSource:0}: Error finding container acfffea194d3433763118522b1dcab8ab92764d16452ae9333cffbb600fbbf65: Status 404 returned error can't find the container with id acfffea194d3433763118522b1dcab8ab92764d16452ae9333cffbb600fbbf65
Sep 11 11:34:22.954240 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:22.954217    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod313b9fe9_a1e5_4f5a_af78_1bffd2ab7329.slice/crio-90d0e4853c8b6fb729a2d1448f975e9b102011bd0de65ceb7da7fcef19a49c84 WatchSource:0}: Error finding container 90d0e4853c8b6fb729a2d1448f975e9b102011bd0de65ceb7da7fcef19a49c84: Status 404 returned error can't find the container with id 90d0e4853c8b6fb729a2d1448f975e9b102011bd0de65ceb7da7fcef19a49c84
Sep 11 11:34:22.954967 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:34:22.954952    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8608f214_d97e_496d_936e_fdc852376d73.slice/crio-e0dbd75157977d3a98affd9a7d82909b5148d7809f92d0b8e3c7fbc2ae46212b WatchSource:0}: Error finding container e0dbd75157977d3a98affd9a7d82909b5148d7809f92d0b8e3c7fbc2ae46212b: Status 404 returned error can't find the container with id e0dbd75157977d3a98affd9a7d82909b5148d7809f92d0b8e3c7fbc2ae46212b
Sep 11 11:34:22.988344 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.988228    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 11:29:21 +0000 UTC" deadline="2028-04-23 11:30:39.800929539 +0000 UTC"
Sep 11 11:34:22.988344 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:22.988344    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14159h56m16.812588737s"
Sep 11 11:34:23.075434 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:23.075409    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:23.075531 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:23.075501    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:34:23.084533 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:23.084510    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tc2q8" event={"ID":"0ee3a4a6-522e-4174-a9aa-5da4e429d371","Type":"ContainerStarted","Data":"77ea0bc83bf127cd26293683205f79e9ca0a49fdfa3879531f1f537664b1df8d"}
Sep 11 11:34:23.085496 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:23.085473    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vf4g9" event={"ID":"88a0717f-18e4-40d9-914c-96ccaaf0e05f","Type":"ContainerStarted","Data":"15a02e566339c3735e116a4775aefad9674435f31b096936bcbe276a401c5cab"}
Sep 11 11:34:23.086465 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:23.086447    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-flt4d" event={"ID":"9740ecb1-761c-45bc-a8c1-866c71e5c32e","Type":"ContainerStarted","Data":"9ea4c6b5a29d1c18481f49997050c4d43bd7d93857682e54f048d314977963f3"}
Sep 11 11:34:23.087675 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:23.087655    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x7p9s" event={"ID":"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329","Type":"ContainerStarted","Data":"90d0e4853c8b6fb729a2d1448f975e9b102011bd0de65ceb7da7fcef19a49c84"}
Sep 11 11:34:23.088843 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:23.088820    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lx4vv" event={"ID":"9470c69c-fd20-49fc-8b0f-e02d3010e1bc","Type":"ContainerStarted","Data":"acfffea194d3433763118522b1dcab8ab92764d16452ae9333cffbb600fbbf65"}
Sep 11 11:34:23.089889 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:23.089872    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6tqm4" event={"ID":"412f846c-0794-451d-b9ed-300409e19f26","Type":"ContainerStarted","Data":"ac37c8620b03fe53d792b00fb02a9efd6c38b98a52bafa2113969dc0079606e9"}
Sep 11 11:34:23.094950 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:23.094932    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb" event={"ID":"72a9a805-b878-45fd-a7d6-3306117d1975","Type":"ContainerStarted","Data":"38ff4b5f4b6961e7bdcbd228b528b1db95d97a996bff5377843a9fe4a6c83be7"}
Sep 11 11:34:23.096822 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:23.096800    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc" event={"ID":"16e07918-992a-4e66-a4c1-e203cc201788","Type":"ContainerStarted","Data":"5a1617c786d50beb108f5ccf09b0c36dd59f869942435d7e895946c722df16ed"}
Sep 11 11:34:23.098936 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:23.098919    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-16.ec2.internal" event={"ID":"a7293eba94400e360077848fce69243b","Type":"ContainerStarted","Data":"db3d0294156d8ad48480dd2f2e98e06c9643ad40613940c25fb4e09be6b3839e"}
Sep 11 11:34:23.100103 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:23.100083    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-679fd" event={"ID":"8608f214-d97e-496d-936e-fdc852376d73","Type":"ContainerStarted","Data":"e0dbd75157977d3a98affd9a7d82909b5148d7809f92d0b8e3c7fbc2ae46212b"}
Sep 11 11:34:23.115353 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:23.115311    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-16.ec2.internal" podStartSLOduration=2.115297204 podStartE2EDuration="2.115297204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:34:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:34:23.115109326 +0000 UTC m=+3.535715359" watchObservedRunningTime="2026-09-11 11:34:23.115297204 +0000 UTC m=+3.535903229"
Sep 11 11:34:23.568384 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:23.567802    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs\") pod \"network-metrics-daemon-2tbn7\" (UID: \"7d81bb62-6b31-4915-8827-f8d7347665c0\") " pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:23.568384 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:23.567969    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:34:23.568384 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:23.568036    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs podName:7d81bb62-6b31-4915-8827-f8d7347665c0 nodeName:}" failed. No retries permitted until 2026-09-11 11:34:25.568015192 +0000 UTC m=+5.988621220 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs") pod "network-metrics-daemon-2tbn7" (UID: "7d81bb62-6b31-4915-8827-f8d7347665c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:34:23.669085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:23.668729    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret\") pod \"global-pull-secret-syncer-nhrgs\" (UID: \"e58ed647-be4f-4e08-b5e9-d919687aec38\") " pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:23.669085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:23.668801    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmxn7\" (UniqueName: \"kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7\") pod \"network-check-target-wklxl\" (UID: \"0b9f399e-7383-4c4a-897c-fa71548f24d1\") " pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:23.669085 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:23.668936    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:34:23.669085 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:23.668956    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:34:23.669085 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:23.668968    2575 projected.go:196] Error preparing data for projected volume kube-api-access-rmxn7 for pod openshift-network-diagnostics/network-check-target-wklxl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:34:23.669085 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:23.668998    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:34:23.669085 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:23.669026    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7 podName:0b9f399e-7383-4c4a-897c-fa71548f24d1 nodeName:}" failed. No retries permitted until 2026-09-11 11:34:25.669008447 +0000 UTC m=+6.089614473 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rmxn7" (UniqueName: "kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7") pod "network-check-target-wklxl" (UID: "0b9f399e-7383-4c4a-897c-fa71548f24d1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:34:23.669085 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:23.669043    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret podName:e58ed647-be4f-4e08-b5e9-d919687aec38 nodeName:}" failed. No retries permitted until 2026-09-11 11:34:25.669035054 +0000 UTC m=+6.089641059 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret") pod "global-pull-secret-syncer-nhrgs" (UID: "e58ed647-be4f-4e08-b5e9-d919687aec38") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:34:23.989946 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:23.989849    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 11:29:21 +0000 UTC" deadline="2028-05-25 03:47:00.628567474 +0000 UTC"
Sep 11 11:34:23.990523 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:23.989948    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14920h12m36.638624362s"
Sep 11 11:34:24.084118 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:24.084089    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:24.084277 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:24.084203    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:34:24.084596 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:24.084574    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:24.084705 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:24.084686    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:34:24.113217 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:24.113184    2575 generic.go:356] "Generic (PLEG): container finished" podID="e055cf25dcc047127c65d7598806ffba" containerID="dd1efdad5ceae8c896a02a390efb3ae46ce0724c23abf1c202dbed48e98d3a3a" exitCode=0
Sep 11 11:34:24.114303 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:24.114047    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal" event={"ID":"e055cf25dcc047127c65d7598806ffba","Type":"ContainerDied","Data":"dd1efdad5ceae8c896a02a390efb3ae46ce0724c23abf1c202dbed48e98d3a3a"}
Sep 11 11:34:25.075719 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:25.075691    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:25.076168 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:25.075825    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:34:25.132950 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:25.132868    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal" event={"ID":"e055cf25dcc047127c65d7598806ffba","Type":"ContainerStarted","Data":"e2fbc2acd7bb1ffaf51fb1ac4348198bb3c95ddabc61b6d26ac52b2351c91b74"}
Sep 11 11:34:25.149053 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:25.148272    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-16.ec2.internal" podStartSLOduration=4.148252079 podStartE2EDuration="4.148252079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:34:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:34:25.147806766 +0000 UTC m=+5.568412794" watchObservedRunningTime="2026-09-11 11:34:25.148252079 +0000 UTC m=+5.568858107"
Sep 11 11:34:25.584788 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:25.584752    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs\") pod \"network-metrics-daemon-2tbn7\" (UID: \"7d81bb62-6b31-4915-8827-f8d7347665c0\") " pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:25.584967 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:25.584929    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:34:25.585040 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:25.584995    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs podName:7d81bb62-6b31-4915-8827-f8d7347665c0 nodeName:}" failed. No retries permitted until 2026-09-11 11:34:29.584974723 +0000 UTC m=+10.005580746 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs") pod "network-metrics-daemon-2tbn7" (UID: "7d81bb62-6b31-4915-8827-f8d7347665c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:34:25.685412 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:25.685377    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret\") pod \"global-pull-secret-syncer-nhrgs\" (UID: \"e58ed647-be4f-4e08-b5e9-d919687aec38\") " pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:25.685589 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:25.685451    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmxn7\" (UniqueName: \"kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7\") pod \"network-check-target-wklxl\" (UID: \"0b9f399e-7383-4c4a-897c-fa71548f24d1\") " pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:25.685765 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:25.685613    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:34:25.685765 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:25.685651    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:34:25.685765 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:25.685664    2575 projected.go:196] Error preparing data for projected volume kube-api-access-rmxn7 for pod openshift-network-diagnostics/network-check-target-wklxl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:34:25.685765 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:25.685734    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7 podName:0b9f399e-7383-4c4a-897c-fa71548f24d1 nodeName:}" failed. No retries permitted until 2026-09-11 11:34:29.68571299 +0000 UTC m=+10.106319016 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rmxn7" (UniqueName: "kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7") pod "network-check-target-wklxl" (UID: "0b9f399e-7383-4c4a-897c-fa71548f24d1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:34:25.685765 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:25.685763    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:34:25.686038 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:25.685829    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret podName:e58ed647-be4f-4e08-b5e9-d919687aec38 nodeName:}" failed. No retries permitted until 2026-09-11 11:34:29.685810677 +0000 UTC m=+10.106416887 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret") pod "global-pull-secret-syncer-nhrgs" (UID: "e58ed647-be4f-4e08-b5e9-d919687aec38") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:34:26.077848 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:26.077767    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:26.078294 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:26.077874    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:34:26.078294 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:26.077921    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:26.078294 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:26.077965    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:34:27.075201 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:27.075168    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:27.075394 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:27.075318    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:34:28.077772 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:28.077743    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:28.078217 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:28.077862    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:34:28.078217 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:28.077992    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:28.078217 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:28.078075    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:34:29.076077 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:29.076047    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:29.076273 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:29.076197    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:34:29.619518 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:29.619465    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs\") pod \"network-metrics-daemon-2tbn7\" (UID: \"7d81bb62-6b31-4915-8827-f8d7347665c0\") " pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:29.620236 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:29.619656    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:34:29.620236 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:29.619726    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs podName:7d81bb62-6b31-4915-8827-f8d7347665c0 nodeName:}" failed. No retries permitted until 2026-09-11 11:34:37.619707161 +0000 UTC m=+18.040313168 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs") pod "network-metrics-daemon-2tbn7" (UID: "7d81bb62-6b31-4915-8827-f8d7347665c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:34:29.720712 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:29.720671    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmxn7\" (UniqueName: \"kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7\") pod \"network-check-target-wklxl\" (UID: \"0b9f399e-7383-4c4a-897c-fa71548f24d1\") " pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:29.720881 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:29.720755    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret\") pod \"global-pull-secret-syncer-nhrgs\" (UID: \"e58ed647-be4f-4e08-b5e9-d919687aec38\") " pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:29.720881 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:29.720848    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:34:29.720881 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:29.720853    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:34:29.720881 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:29.720874    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:34:29.721077 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:29.720886    2575 projected.go:196] Error preparing data for projected volume kube-api-access-rmxn7 for pod openshift-network-diagnostics/network-check-target-wklxl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:34:29.721077 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:29.720893    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret podName:e58ed647-be4f-4e08-b5e9-d919687aec38 nodeName:}" failed. No retries permitted until 2026-09-11 11:34:37.72087885 +0000 UTC m=+18.141484855 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret") pod "global-pull-secret-syncer-nhrgs" (UID: "e58ed647-be4f-4e08-b5e9-d919687aec38") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:34:29.721077 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:29.720935    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7 podName:0b9f399e-7383-4c4a-897c-fa71548f24d1 nodeName:}" failed. No retries permitted until 2026-09-11 11:34:37.72091841 +0000 UTC m=+18.141524437 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rmxn7" (UniqueName: "kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7") pod "network-check-target-wklxl" (UID: "0b9f399e-7383-4c4a-897c-fa71548f24d1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:34:30.076721 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:30.076537    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:30.076721 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:30.076568    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:30.076721 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:30.076672    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:34:30.077108 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:30.077067    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:34:31.075942 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:31.075910    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:31.076369 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:31.076044    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:34:32.075773 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:32.075732    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:32.075956 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:32.075880    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:34:32.075956 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:32.075933    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:32.076454 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:32.076034    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:34:33.075716 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:33.075674    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:33.075896 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:33.075815    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:34:34.076080 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:34.076050    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:34.076519 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:34.076065    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:34.076519 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:34.076177    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:34:34.076519 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:34.076261    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:34:35.075548 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:35.075512    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:35.075737 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:35.075667    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:34:36.075354 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:36.075319    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:36.075789 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:36.075327    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:36.075789 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:36.075433    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:34:36.075789 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:36.075529    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:34:37.075844 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:37.075770    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:37.076258 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:37.075910    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:34:37.678280 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:37.678249    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs\") pod \"network-metrics-daemon-2tbn7\" (UID: \"7d81bb62-6b31-4915-8827-f8d7347665c0\") " pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:37.678462 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:37.678400    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:34:37.678528 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:37.678470    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs podName:7d81bb62-6b31-4915-8827-f8d7347665c0 nodeName:}" failed. No retries permitted until 2026-09-11 11:34:53.6784552 +0000 UTC m=+34.099061208 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs") pod "network-metrics-daemon-2tbn7" (UID: "7d81bb62-6b31-4915-8827-f8d7347665c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:34:37.779239 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:37.779206    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret\") pod \"global-pull-secret-syncer-nhrgs\" (UID: \"e58ed647-be4f-4e08-b5e9-d919687aec38\") " pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:37.779405 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:37.779274    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmxn7\" (UniqueName: \"kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7\") pod \"network-check-target-wklxl\" (UID: \"0b9f399e-7383-4c4a-897c-fa71548f24d1\") " pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:37.779405 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:37.779377    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:34:37.779533 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:37.779407    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:34:37.779533 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:37.779427    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:34:37.779533 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:37.779439    2575 projected.go:196] Error preparing data for projected volume kube-api-access-rmxn7 for pod openshift-network-diagnostics/network-check-target-wklxl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:34:37.779533 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:37.779449    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret podName:e58ed647-be4f-4e08-b5e9-d919687aec38 nodeName:}" failed. No retries permitted until 2026-09-11 11:34:53.779428188 +0000 UTC m=+34.200034208 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret") pod "global-pull-secret-syncer-nhrgs" (UID: "e58ed647-be4f-4e08-b5e9-d919687aec38") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:34:37.779533 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:37.779513    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7 podName:0b9f399e-7383-4c4a-897c-fa71548f24d1 nodeName:}" failed. No retries permitted until 2026-09-11 11:34:53.779469809 +0000 UTC m=+34.200075818 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rmxn7" (UniqueName: "kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7") pod "network-check-target-wklxl" (UID: "0b9f399e-7383-4c4a-897c-fa71548f24d1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:34:38.076025 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:38.075949    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:38.076402 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:38.075948    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:38.076402 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:38.076086    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:34:38.076402 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:38.076168    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:34:39.075333 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:39.075294    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:39.075506 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:39.075423    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:34:40.076506 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:40.076464    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:40.076925 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:40.076571    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:34:40.076925 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:40.076646    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:40.076925 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:40.076785    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:34:41.076650 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:41.075495    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:41.076650 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:41.075858    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:34:41.173295 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:41.172405    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tc2q8" event={"ID":"0ee3a4a6-522e-4174-a9aa-5da4e429d371","Type":"ContainerStarted","Data":"ec595c4d7faa1109fd01dda31599faacc18331e6d6200c8ee0b9f037b4bbefed"}
Sep 11 11:34:41.178743 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:41.178312    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vf4g9" event={"ID":"88a0717f-18e4-40d9-914c-96ccaaf0e05f","Type":"ContainerStarted","Data":"a01ee7e8417c6343002933ed8668a0100ac5c0e28382c9c7e71f789348718cd6"}
Sep 11 11:34:41.182242 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:41.181894    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-flt4d" event={"ID":"9740ecb1-761c-45bc-a8c1-866c71e5c32e","Type":"ContainerStarted","Data":"0a9cb2e1a70649fc73a13febd0d8533d82e255b646202754abb5d1ac0569dd9d"}
Sep 11 11:34:41.184762 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:41.184731    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x7p9s" event={"ID":"313b9fe9-a1e5-4f5a-af78-1bffd2ab7329","Type":"ContainerStarted","Data":"7ef106ddb2bd7edc0d85c51f3c4e3713448252f1befc06b5e5fb5b93cedd6159"}
Sep 11 11:34:41.186360 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:41.186336    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lx4vv" event={"ID":"9470c69c-fd20-49fc-8b0f-e02d3010e1bc","Type":"ContainerStarted","Data":"f20e12bb53a7bc4ab6a8bd87a22a7ea6b71e753fec77bcdbad20121054b5f07a"}
Sep 11 11:34:41.188269 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:41.188253    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6tqm4" event={"ID":"412f846c-0794-451d-b9ed-300409e19f26","Type":"ContainerStarted","Data":"cf8ddf09b4730cbe1e7ba91811969e6f6927d8cc7a58ef348192dcf014482d37"}
Sep 11 11:34:41.190729 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:41.190594    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb" event={"ID":"72a9a805-b878-45fd-a7d6-3306117d1975","Type":"ContainerStarted","Data":"a22d7f443e4ad07d7caf23c96928344440722dcee66cc6dd2510155395c7a369"}
Sep 11 11:34:41.190729 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:41.190699    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb" event={"ID":"72a9a805-b878-45fd-a7d6-3306117d1975","Type":"ContainerStarted","Data":"39ce3d7796a4d1386a73cb02be60ac148291510063b9d437605f5f6cd8540b79"}
Sep 11 11:34:41.190897 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:41.190795    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tc2q8" podStartSLOduration=3.182519996 podStartE2EDuration="21.190782267s" totalImagesPullingTime="18.008262271s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:34:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:34:41.190529401 +0000 UTC m=+21.611135425" watchObservedRunningTime="2026-09-11 11:34:41.190782267 +0000 UTC m=+21.611388304"
Sep 11 11:34:41.194450 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:41.194425    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc" event={"ID":"16e07918-992a-4e66-a4c1-e203cc201788","Type":"ContainerStarted","Data":"194fed9a85835dbcb8dc47af2f42ebadf5361df07b36be3944dfe692c998a96e"}
Sep 11 11:34:41.206435 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:41.206394    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vf4g9" podStartSLOduration=3.236045141 podStartE2EDuration="21.20638032s" totalImagesPullingTime="17.970335179s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:34:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:34:41.205803897 +0000 UTC m=+21.626409915" watchObservedRunningTime="2026-09-11 11:34:41.20638032 +0000 UTC m=+21.626986344"
Sep 11 11:34:41.248595 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:41.248558    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-x7p9s" podStartSLOduration=3.301472439 podStartE2EDuration="21.248543535s" totalImagesPullingTime="17.947071096s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:34:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:34:41.248232034 +0000 UTC m=+21.668838073" watchObservedRunningTime="2026-09-11 11:34:41.248543535 +0000 UTC m=+21.669149553"
Sep 11 11:34:41.266215 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:41.266177    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6tqm4" podStartSLOduration=7.439152986 podStartE2EDuration="21.266162161s" totalImagesPullingTime="13.827009175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:34:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:34:41.265817922 +0000 UTC m=+21.686423948" watchObservedRunningTime="2026-09-11 11:34:41.266162161 +0000 UTC m=+21.686768187"
Sep 11 11:34:41.286150 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:41.286113    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lx4vv" podStartSLOduration=4.371135733 podStartE2EDuration="22.286098974s" totalImagesPullingTime="17.914963241s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:34:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:34:41.285457075 +0000 UTC m=+21.706063101" watchObservedRunningTime="2026-09-11 11:34:41.286098974 +0000 UTC m=+21.706705053"
Sep 11 11:34:41.371603 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:41.371579    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:41.446931 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:41.446905    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:41.548208 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:41.548183    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:41.617424 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:41.617394    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:42.076119 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:42.076092    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:42.076271 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:42.076133    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:42.076271 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:42.076212    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:34:42.076380 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:42.076355    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:34:42.199749 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:42.199716    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb" event={"ID":"72a9a805-b878-45fd-a7d6-3306117d1975","Type":"ContainerStarted","Data":"b76952bbab2064e683ef10b477608db8573237c6517f3fa0a374243a477fad23"}
Sep 11 11:34:42.200215 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:42.199780    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb" event={"ID":"72a9a805-b878-45fd-a7d6-3306117d1975","Type":"ContainerStarted","Data":"1c6c671e98431feee341a9340e047b6edfd1e9c0c3dabba74d6e77480b79ee97"}
Sep 11 11:34:42.200215 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:42.199816    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb" event={"ID":"72a9a805-b878-45fd-a7d6-3306117d1975","Type":"ContainerStarted","Data":"8cfc1001c1967476a628ea13d2694359870607f9c2f1c20a984e8ab52dbb912d"}
Sep 11 11:34:42.200215 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:42.199848    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb" event={"ID":"72a9a805-b878-45fd-a7d6-3306117d1975","Type":"ContainerStarted","Data":"2b3c26cc33dea06eff319ac04a2c5ea3c29ad7fe0a5d6c441f183b5bf187455b"}
Sep 11 11:34:42.201118 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:42.201094    2575 generic.go:356] "Generic (PLEG): container finished" podID="9740ecb1-761c-45bc-a8c1-866c71e5c32e" containerID="0a9cb2e1a70649fc73a13febd0d8533d82e255b646202754abb5d1ac0569dd9d" exitCode=0
Sep 11 11:34:42.201220 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:42.201189    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-flt4d" event={"ID":"9740ecb1-761c-45bc-a8c1-866c71e5c32e","Type":"ContainerDied","Data":"0a9cb2e1a70649fc73a13febd0d8533d82e255b646202754abb5d1ac0569dd9d"}
Sep 11 11:34:42.241554 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:42.241522    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:42.340243 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:42.340222    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:42.412515 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:42.412474    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:42.444981 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:42.444958    2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 11:34:42.831249 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:42.831188    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:42.931367 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:42.931329    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:43.017407 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:43.017302    2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T11:34:42.444978825Z","UUID":"2e48b148-03f4-4ccd-a56b-00e257b6502b","Handler":null,"Name":"","Endpoint":""}
Sep 11 11:34:43.019137 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:43.019112    2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 11:34:43.019277 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:43.019149    2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 11:34:43.076057 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:43.076024    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:43.076240 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:43.076176    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:34:43.098489 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:43.098420    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:43.181263 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:43.181228    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:43.205975 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:43.205933    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc" event={"ID":"16e07918-992a-4e66-a4c1-e203cc201788","Type":"ContainerStarted","Data":"74869dac061aa821db011fc5bd79089656d2dc2cf595ed610ff4d4d465bcaf2f"}
Sep 11 11:34:43.207550 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:43.207524    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-679fd" event={"ID":"8608f214-d97e-496d-936e-fdc852376d73","Type":"ContainerStarted","Data":"de7262fec025a5c8b4d94f9e8b19ba47f7f81a7aa1439221315f69a78666ab34"}
Sep 11 11:34:43.224252 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:43.224175    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-679fd" podStartSLOduration=5.278729718 podStartE2EDuration="23.224159904s" totalImagesPullingTime="17.945430186s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:34:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:34:43.223758092 +0000 UTC m=+23.644364119" watchObservedRunningTime="2026-09-11 11:34:43.224159904 +0000 UTC m=+23.644765931"
Sep 11 11:34:43.321504 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:43.321467    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:43.387414 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:43.387330    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:44.075823 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:44.075795    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:44.075823 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:44.075811    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:44.076052 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:44.075902    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:34:44.076120 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:44.076044    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:34:44.213964 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:44.213930    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb" event={"ID":"72a9a805-b878-45fd-a7d6-3306117d1975","Type":"ContainerStarted","Data":"2566f8ab2968337b68f50951bc8b49a9e83468c0dcec70a1cdcd4e0d299d8c30"}
Sep 11 11:34:44.216291 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:44.216265    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc" event={"ID":"16e07918-992a-4e66-a4c1-e203cc201788","Type":"ContainerStarted","Data":"29cf9174d02bf88dbe890d337452c800992f97ed11065b6335501f955f3939e3"}
Sep 11 11:34:44.242094 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:44.242056    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djnzc" podStartSLOduration=4.036807749 podStartE2EDuration="24.242044215s" totalImagesPullingTime="20.205236466s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:34:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:34:44.241900655 +0000 UTC m=+24.662506682" watchObservedRunningTime="2026-09-11 11:34:44.242044215 +0000 UTC m=+24.662650241"
Sep 11 11:34:44.247545 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:44.247516    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:44.338661 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:44.338569    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:44.730005 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:44.729923    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:44.828891 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:44.828860    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:44.924170 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:44.924146    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vf4g9_88a0717f-18e4-40d9-914c-96ccaaf0e05f/dns-node-resolver/0.log"
Sep 11 11:34:45.075620 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:45.075591    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:45.075802 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:45.075758    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:34:45.080704 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:45.080677    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6tqm4"
Sep 11 11:34:45.081295 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:45.081274    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6tqm4"
Sep 11 11:34:45.707837 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:45.707617    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:46.075741 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:46.075709    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:46.075934 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:46.075838    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:34:46.075934 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:46.075906    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:46.076059 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:46.076017    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:34:46.124263 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:46.124234    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lx4vv_9470c69c-fd20-49fc-8b0f-e02d3010e1bc/node-ca/0.log"
Sep 11 11:34:47.075061 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:47.075026    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:47.075480 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:47.075134    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:34:47.226220 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:47.226183    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb" event={"ID":"72a9a805-b878-45fd-a7d6-3306117d1975","Type":"ContainerStarted","Data":"316b55a720d8b296d714fc989b14219e7b23103a8378860ba917d5d93a740a2f"}
Sep 11 11:34:47.226558 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:47.226538    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:47.226658 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:47.226650    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:47.231601 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:47.231564    2575 generic.go:356] "Generic (PLEG): container finished" podID="9740ecb1-761c-45bc-a8c1-866c71e5c32e" containerID="5cf7f61a7bee5e98d36ecb68530b361368fde00cc048178e1dc4c0b6f94a06c2" exitCode=0
Sep 11 11:34:47.231601 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:47.231599    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-flt4d" event={"ID":"9740ecb1-761c-45bc-a8c1-866c71e5c32e","Type":"ContainerDied","Data":"5cf7f61a7bee5e98d36ecb68530b361368fde00cc048178e1dc4c0b6f94a06c2"}
Sep 11 11:34:47.242970 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:47.242951    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:47.260623 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:47.260588    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb" podStartSLOduration=9.242238983 podStartE2EDuration="27.26057706s" totalImagesPullingTime="18.018338077s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:34:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:34:47.260132973 +0000 UTC m=+27.680738999" watchObservedRunningTime="2026-09-11 11:34:47.26057706 +0000 UTC m=+27.681183087"
Sep 11 11:34:48.075858 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:48.075827    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:48.076218 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:48.075838    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:48.076218 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:48.075925    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:34:48.076218 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:48.075996    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:34:48.233684 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:48.233657    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:48.248471 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:48.248447    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:34:48.924925 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:48.924726    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:49.075005 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:49.074979    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:49.075139 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:49.075077    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:34:49.237050 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:49.237020    2575 generic.go:356] "Generic (PLEG): container finished" podID="9740ecb1-761c-45bc-a8c1-866c71e5c32e" containerID="526471eb391c545a89c003c62481f5ad3539fab4a4199bca1e41732d6d012082" exitCode=0
Sep 11 11:34:49.237484 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:49.237082    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-flt4d" event={"ID":"9740ecb1-761c-45bc-a8c1-866c71e5c32e","Type":"ContainerDied","Data":"526471eb391c545a89c003c62481f5ad3539fab4a4199bca1e41732d6d012082"}
Sep 11 11:34:49.315100 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:49.315066    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:49.383027 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:49.383000    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:49.699306 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:49.699237    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:50.013726 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:50.013690    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:50.076387 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:50.076362    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:50.076525 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:50.076436    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:34:50.076574 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:50.076528    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:50.076627 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:50.076609    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:34:50.260436 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:50.260411    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:50.449957 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:50.449931    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:50.556973 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:50.556912    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:50.757947 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:50.757913    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:51.075514 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:51.075487    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:51.075692 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:51.075586    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:34:51.242935 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:51.242902    2575 generic.go:356] "Generic (PLEG): container finished" podID="9740ecb1-761c-45bc-a8c1-866c71e5c32e" containerID="bcf2f013c93963fb80998cdc5156aace9dadf721661046bb83a288a469beb632" exitCode=0
Sep 11 11:34:51.243065 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:51.242949    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-flt4d" event={"ID":"9740ecb1-761c-45bc-a8c1-866c71e5c32e","Type":"ContainerDied","Data":"bcf2f013c93963fb80998cdc5156aace9dadf721661046bb83a288a469beb632"}
Sep 11 11:34:51.372458 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:51.372391    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6tqm4"
Sep 11 11:34:51.372866 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:51.372521    2575 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 11 11:34:51.372986 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:51.372971    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6tqm4"
Sep 11 11:34:51.718534 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:51.718448    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:51.939548 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:51.939512    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:52.075874 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:52.075844    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:52.076038 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:52.075845    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:52.076038 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:52.075951    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:34:52.076038 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:52.076029    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:34:52.116156 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:52.116121    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:52.361212 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:52.361134    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:52.526554 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:52.526523    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:52.730073 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:52.729997    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:52.922891 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:52.922860    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:53.075792 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:53.075765    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:53.075936 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:53.075870    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:34:53.653027 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:53.652994    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:53.706494 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:53.706471    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs\") pod \"network-metrics-daemon-2tbn7\" (UID: \"7d81bb62-6b31-4915-8827-f8d7347665c0\") " pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:53.706688 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:53.706652    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:34:53.706758 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:53.706717    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs podName:7d81bb62-6b31-4915-8827-f8d7347665c0 nodeName:}" failed. No retries permitted until 2026-09-11 11:35:25.706702861 +0000 UTC m=+66.127308886 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs") pod "network-metrics-daemon-2tbn7" (UID: "7d81bb62-6b31-4915-8827-f8d7347665c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 11:34:53.756757 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:53.756728    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:53.807751 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:53.807721    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmxn7\" (UniqueName: \"kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7\") pod \"network-check-target-wklxl\" (UID: \"0b9f399e-7383-4c4a-897c-fa71548f24d1\") " pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:53.807887 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:53.807780    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret\") pod \"global-pull-secret-syncer-nhrgs\" (UID: \"e58ed647-be4f-4e08-b5e9-d919687aec38\") " pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:53.807887 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:53.807863    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 11:34:53.807887 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:53.807881    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 11:34:53.807887 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:53.807890    2575 projected.go:196] Error preparing data for projected volume kube-api-access-rmxn7 for pod openshift-network-diagnostics/network-check-target-wklxl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:34:53.808092 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:53.807898    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 11:34:53.808092 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:53.807937    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7 podName:0b9f399e-7383-4c4a-897c-fa71548f24d1 nodeName:}" failed. No retries permitted until 2026-09-11 11:35:25.807924047 +0000 UTC m=+66.228530055 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rmxn7" (UniqueName: "kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7") pod "network-check-target-wklxl" (UID: "0b9f399e-7383-4c4a-897c-fa71548f24d1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 11:34:53.808092 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:53.807952    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret podName:e58ed647-be4f-4e08-b5e9-d919687aec38 nodeName:}" failed. No retries permitted until 2026-09-11 11:35:25.807944995 +0000 UTC m=+66.228551003 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret") pod "global-pull-secret-syncer-nhrgs" (UID: "e58ed647-be4f-4e08-b5e9-d919687aec38") : object "kube-system"/"original-pull-secret" not registered
Sep 11 11:34:53.854243 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:53.854212    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:53.954429 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:53.954344    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:54.075881 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:54.075851    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:54.076054 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:54.075984    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:34:54.076117 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:54.076058    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:54.076167 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:54.076142    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:34:54.119779 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:54.119748    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:54.312910 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:54.312867    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:54.382425 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:54.382375    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:54.522735 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:54.522706    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:54.593778 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:54.593674    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:54.930497 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:54.930425    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:55.075195 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:55.075168    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:55.075378 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:55.075281    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:34:55.520797 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:55.520761    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:55.622020 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:55.621984    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:55.743574 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:55.743541    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:55.926759 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:55.926727    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:56.075595 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:56.075539    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:56.075921 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:56.075684    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:34:56.075921 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:56.075719    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:56.075921 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:56.075816    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:34:56.117212 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:56.117184    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:56.242872 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:56.242842    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:56.257153 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:56.257127    2575 generic.go:356] "Generic (PLEG): container finished" podID="9740ecb1-761c-45bc-a8c1-866c71e5c32e" containerID="b6c03deaecc284c275c5e8f57a601fe195dca1b732a9796c4cf716c46df66bf1" exitCode=0
Sep 11 11:34:56.257279 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:56.257190    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-flt4d" event={"ID":"9740ecb1-761c-45bc-a8c1-866c71e5c32e","Type":"ContainerDied","Data":"b6c03deaecc284c275c5e8f57a601fe195dca1b732a9796c4cf716c46df66bf1"}
Sep 11 11:34:56.454975 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:56.454950    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:56.556331 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:56.556306    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:57.075942 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:57.075906    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:57.076475 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:57.076035    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:34:57.262084 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:57.262056    2575 generic.go:356] "Generic (PLEG): container finished" podID="9740ecb1-761c-45bc-a8c1-866c71e5c32e" containerID="4af9b22352c2165bb630189e11ddc139e3dda09439d820e5adc7b0bf17e99a13" exitCode=0
Sep 11 11:34:57.262287 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:57.262117    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-flt4d" event={"ID":"9740ecb1-761c-45bc-a8c1-866c71e5c32e","Type":"ContainerDied","Data":"4af9b22352c2165bb630189e11ddc139e3dda09439d820e5adc7b0bf17e99a13"}
Sep 11 11:34:57.321015 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:57.320944    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:57.524293 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:57.524261    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:57.911863 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:57.911831    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:58.075795 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:58.075768    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:34:58.075947 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:58.075857    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:34:58.075947 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:58.075894    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:34:58.076266 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:58.075947    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:34:58.123924 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:58.123891    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:58.267104 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:58.267081    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-flt4d" event={"ID":"9740ecb1-761c-45bc-a8c1-866c71e5c32e","Type":"ContainerStarted","Data":"3ae190217fd5ba80629bf023bcfc654a903464ea586976d67e611228c23bec7d"}
Sep 11 11:34:58.520339 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:58.520260    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:59.075332 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:59.075302    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:34:59.075480 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:34:59.075404    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:34:59.118144 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:59.118117    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:59.730699 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:59.730664    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:34:59.910380 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:34:59.910354    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:00.075788 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:00.075765    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:35:00.075952 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:00.075847    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:35:00.075952 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:00.075936    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:35:00.076024 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:00.076006    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:35:00.312753 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:00.312726    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:00.538729 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:00.538707    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:00.923797 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:00.923708    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:01.075946 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:01.075914    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:35:01.076355 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:01.076323    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:35:01.183063 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:01.182988    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:02.075951 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:02.075919    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:35:02.075951 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:02.075935    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:35:02.076420 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:02.076017    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:35:02.076420 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:02.076159    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:35:02.749478 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:02.749441    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:03.075862 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:03.075792    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:35:03.076003 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:03.075887    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:35:03.971021 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:03.970986    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:04.048320 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:04.048295    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:04.075255 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:04.075234    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:35:04.075343 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:04.075234    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:35:04.075383 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:04.075339    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:35:04.075417 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:04.075403    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:35:05.075145 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:05.075114    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:35:05.075607 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:05.075219    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:35:06.075953 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:06.075930    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:35:06.075953 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:06.075944    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:35:06.076375 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:06.076018    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:35:06.076375 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:06.076152    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:35:07.075063 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:07.075031    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:35:07.075230 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:07.075155    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:35:08.075304 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:08.075270    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:35:08.075751 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:08.075382    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:35:08.075751 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:08.075429    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:35:08.075751 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:08.075519    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:35:08.161250 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:08.161197    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-flt4d" podStartSLOduration=18.890216697 podStartE2EDuration="48.161180088s" totalImagesPullingTime="29.270963391s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:34:20 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:34:58.292239913 +0000 UTC m=+38.712845941" watchObservedRunningTime="2026-09-11 11:35:08.161180088 +0000 UTC m=+48.581786114"
Sep 11 11:35:08.253328 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:08.253297    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:08.370775 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:08.370687    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:08.516054 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:08.516021    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:08.615036 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:08.615005    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:08.719316 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:08.719238    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:08.816448 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:08.816410    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:08.955437 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:08.955397    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:09.076028 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:09.076000    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:35:09.076435 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:09.076111    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:35:09.283784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:09.283581    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:09.607725 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:09.607695    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nhrgs"]
Sep 11 11:35:09.607877 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:09.607800    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:35:09.607942 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:09.607902    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:35:09.611614 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:09.611577    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wklxl"]
Sep 11 11:35:09.611914 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:09.611898    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:35:09.612139 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:09.612107    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:35:09.612285 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:09.612187    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2tbn7"]
Sep 11 11:35:09.612285 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:09.612272    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:35:09.612407 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:09.612350    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:35:10.204438 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:10.204393    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:10.284523 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:10.284486    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:10.391882 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:10.391850    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:10.476489 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:10.476412    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:10.843224 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:10.843180    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:10.944327 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:10.944299    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:11.075046 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:11.075016    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:35:11.075186 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:11.075118    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:35:11.075186 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:11.075130    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:35:11.075186 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:11.075131    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:35:11.075303 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:11.075198    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:35:11.075303 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:11.075271    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:35:12.943071 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:12.943037    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:13.008388 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.008353    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:13.075770 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.075740    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:35:13.075932 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.075827    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:35:13.075932 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:13.075833    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nhrgs" podUID="e58ed647-be4f-4e08-b5e9-d919687aec38"
Sep 11 11:35:13.075932 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.075839    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:35:13.075932 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:13.075917    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wklxl" podUID="0b9f399e-7383-4c4a-897c-fa71548f24d1"
Sep 11 11:35:13.076146 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:13.075997    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2tbn7" podUID="7d81bb62-6b31-4915-8827-f8d7347665c0"
Sep 11 11:35:13.602961 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.602926    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:13.658797 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.658767    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:13.863965 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.863934    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-16.ec2.internal" event="NodeReady"
Sep 11 11:35:13.864125 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.864047    2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 11:35:13.910129 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.910094    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wvb2j"]
Sep 11 11:35:13.912035 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.912019    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wvb2j"
Sep 11 11:35:13.914217 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.914198    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-glmqf\""
Sep 11 11:35:13.914335 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.914319    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 11:35:13.914479 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.914461    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 11:35:13.914581 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.914568    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 11:35:13.915761 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.915734    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ngm7d"]
Sep 11 11:35:13.917433 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.917419    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ngm7d"
Sep 11 11:35:13.919722 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.919703    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 11:35:13.919812 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.919706    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9rhw9\""
Sep 11 11:35:13.919993 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.919978    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 11:35:13.920038 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.919997    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 11:35:13.920038 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.920012    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 11:35:13.924732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.924712    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wvb2j"]
Sep 11 11:35:13.929667 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.929644    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ngm7d"]
Sep 11 11:35:13.945314 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.945291    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d070541c-7f68-4350-a203-048ec6f3cf69-crio-socket\") pod \"insights-runtime-extractor-ngm7d\" (UID: \"d070541c-7f68-4350-a203-048ec6f3cf69\") " pod="openshift-insights/insights-runtime-extractor-ngm7d"
Sep 11 11:35:13.945627 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.945327    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d070541c-7f68-4350-a203-048ec6f3cf69-data-volume\") pod \"insights-runtime-extractor-ngm7d\" (UID: \"d070541c-7f68-4350-a203-048ec6f3cf69\") " pod="openshift-insights/insights-runtime-extractor-ngm7d"
Sep 11 11:35:13.945627 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.945362    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29f9b723-dc90-4bf0-bfcb-42bff332e469-cert\") pod \"ingress-canary-wvb2j\" (UID: \"29f9b723-dc90-4bf0-bfcb-42bff332e469\") " pod="openshift-ingress-canary/ingress-canary-wvb2j"
Sep 11 11:35:13.945627 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.945377    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d070541c-7f68-4350-a203-048ec6f3cf69-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ngm7d\" (UID: \"d070541c-7f68-4350-a203-048ec6f3cf69\") " pod="openshift-insights/insights-runtime-extractor-ngm7d"
Sep 11 11:35:13.945627 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.945393    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d070541c-7f68-4350-a203-048ec6f3cf69-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ngm7d\" (UID: \"d070541c-7f68-4350-a203-048ec6f3cf69\") " pod="openshift-insights/insights-runtime-extractor-ngm7d"
Sep 11 11:35:13.945627 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.945457    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhtpt\" (UniqueName: \"kubernetes.io/projected/d070541c-7f68-4350-a203-048ec6f3cf69-kube-api-access-xhtpt\") pod \"insights-runtime-extractor-ngm7d\" (UID: \"d070541c-7f68-4350-a203-048ec6f3cf69\") " pod="openshift-insights/insights-runtime-extractor-ngm7d"
Sep 11 11:35:13.945627 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.945496    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltqvm\" (UniqueName: \"kubernetes.io/projected/29f9b723-dc90-4bf0-bfcb-42bff332e469-kube-api-access-ltqvm\") pod \"ingress-canary-wvb2j\" (UID: \"29f9b723-dc90-4bf0-bfcb-42bff332e469\") " pod="openshift-ingress-canary/ingress-canary-wvb2j"
Sep 11 11:35:13.993330 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:13.993296    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:14.011763 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.011738    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-k6szw"]
Sep 11 11:35:14.013761 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.013746    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-k6szw"
Sep 11 11:35:14.017733 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.017713    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 11:35:14.017817 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.017760    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nnmkw\""
Sep 11 11:35:14.017863 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.017759    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 11:35:14.027586 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.027567    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-k6szw"]
Sep 11 11:35:14.046515 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.046492    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltqvm\" (UniqueName: \"kubernetes.io/projected/29f9b723-dc90-4bf0-bfcb-42bff332e469-kube-api-access-ltqvm\") pod \"ingress-canary-wvb2j\" (UID: \"29f9b723-dc90-4bf0-bfcb-42bff332e469\") " pod="openshift-ingress-canary/ingress-canary-wvb2j"
Sep 11 11:35:14.046611 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.046533    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d070541c-7f68-4350-a203-048ec6f3cf69-crio-socket\") pod \"insights-runtime-extractor-ngm7d\" (UID: \"d070541c-7f68-4350-a203-048ec6f3cf69\") " pod="openshift-insights/insights-runtime-extractor-ngm7d"
Sep 11 11:35:14.046611 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.046557    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d070541c-7f68-4350-a203-048ec6f3cf69-data-volume\") pod \"insights-runtime-extractor-ngm7d\" (UID: \"d070541c-7f68-4350-a203-048ec6f3cf69\") " pod="openshift-insights/insights-runtime-extractor-ngm7d"
Sep 11 11:35:14.046611 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.046579    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf9x5\" (UniqueName: \"kubernetes.io/projected/2561a637-628e-41ca-a07c-18ba49d56cb2-kube-api-access-bf9x5\") pod \"dns-default-k6szw\" (UID: \"2561a637-628e-41ca-a07c-18ba49d56cb2\") " pod="openshift-dns/dns-default-k6szw"
Sep 11 11:35:14.046758 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.046644    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29f9b723-dc90-4bf0-bfcb-42bff332e469-cert\") pod \"ingress-canary-wvb2j\" (UID: \"29f9b723-dc90-4bf0-bfcb-42bff332e469\") " pod="openshift-ingress-canary/ingress-canary-wvb2j"
Sep 11 11:35:14.046758 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.046670    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d070541c-7f68-4350-a203-048ec6f3cf69-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ngm7d\" (UID: \"d070541c-7f68-4350-a203-048ec6f3cf69\") " pod="openshift-insights/insights-runtime-extractor-ngm7d"
Sep 11 11:35:14.046758 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.046696    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d070541c-7f68-4350-a203-048ec6f3cf69-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ngm7d\" (UID: \"d070541c-7f68-4350-a203-048ec6f3cf69\") " pod="openshift-insights/insights-runtime-extractor-ngm7d"
Sep 11 11:35:14.046758 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.046722    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhtpt\" (UniqueName: \"kubernetes.io/projected/d070541c-7f68-4350-a203-048ec6f3cf69-kube-api-access-xhtpt\") pod \"insights-runtime-extractor-ngm7d\" (UID: \"d070541c-7f68-4350-a203-048ec6f3cf69\") " pod="openshift-insights/insights-runtime-extractor-ngm7d"
Sep 11 11:35:14.046758 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.046735    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d070541c-7f68-4350-a203-048ec6f3cf69-crio-socket\") pod \"insights-runtime-extractor-ngm7d\" (UID: \"d070541c-7f68-4350-a203-048ec6f3cf69\") " pod="openshift-insights/insights-runtime-extractor-ngm7d"
Sep 11 11:35:14.046935 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.046746    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2561a637-628e-41ca-a07c-18ba49d56cb2-tmp-dir\") pod \"dns-default-k6szw\" (UID: \"2561a637-628e-41ca-a07c-18ba49d56cb2\") " pod="openshift-dns/dns-default-k6szw"
Sep 11 11:35:14.046935 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.046904    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2561a637-628e-41ca-a07c-18ba49d56cb2-config-volume\") pod \"dns-default-k6szw\" (UID: \"2561a637-628e-41ca-a07c-18ba49d56cb2\") " pod="openshift-dns/dns-default-k6szw"
Sep 11 11:35:14.047021 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.046936    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d070541c-7f68-4350-a203-048ec6f3cf69-data-volume\") pod \"insights-runtime-extractor-ngm7d\" (UID: \"d070541c-7f68-4350-a203-048ec6f3cf69\") " pod="openshift-insights/insights-runtime-extractor-ngm7d"
Sep 11 11:35:14.047021 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.046956    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2561a637-628e-41ca-a07c-18ba49d56cb2-metrics-tls\") pod \"dns-default-k6szw\" (UID: \"2561a637-628e-41ca-a07c-18ba49d56cb2\") " pod="openshift-dns/dns-default-k6szw"
Sep 11 11:35:14.047219 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.047203    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d070541c-7f68-4350-a203-048ec6f3cf69-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ngm7d\" (UID: \"d070541c-7f68-4350-a203-048ec6f3cf69\") " pod="openshift-insights/insights-runtime-extractor-ngm7d"
Sep 11 11:35:14.050863 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.050841    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d070541c-7f68-4350-a203-048ec6f3cf69-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ngm7d\" (UID: \"d070541c-7f68-4350-a203-048ec6f3cf69\") " pod="openshift-insights/insights-runtime-extractor-ngm7d"
Sep 11 11:35:14.050952 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.050906    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-tkcrj"]
Sep 11 11:35:14.050952 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.050938    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29f9b723-dc90-4bf0-bfcb-42bff332e469-cert\") pod \"ingress-canary-wvb2j\" (UID: \"29f9b723-dc90-4bf0-bfcb-42bff332e469\") " pod="openshift-ingress-canary/ingress-canary-wvb2j"
Sep 11 11:35:14.052763 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.052750    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-tkcrj"
Sep 11 11:35:14.055158 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.055139    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-2gl9k\""
Sep 11 11:35:14.055257 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.055216    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 11:35:14.055550 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.055533    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 11:35:14.055696 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.055679    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 11:35:14.055781 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.055623    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 11 11:35:14.058317 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.058288    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 11 11:35:14.060812 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.060792    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltqvm\" (UniqueName: \"kubernetes.io/projected/29f9b723-dc90-4bf0-bfcb-42bff332e469-kube-api-access-ltqvm\") pod \"ingress-canary-wvb2j\" (UID: \"29f9b723-dc90-4bf0-bfcb-42bff332e469\") " pod="openshift-ingress-canary/ingress-canary-wvb2j"
Sep 11 11:35:14.060991 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.060976    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhtpt\" (UniqueName: \"kubernetes.io/projected/d070541c-7f68-4350-a203-048ec6f3cf69-kube-api-access-xhtpt\") pod \"insights-runtime-extractor-ngm7d\" (UID: \"d070541c-7f68-4350-a203-048ec6f3cf69\") " pod="openshift-insights/insights-runtime-extractor-ngm7d"
Sep 11 11:35:14.065937 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.065918    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-tkcrj"]
Sep 11 11:35:14.080738 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.080699    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:14.147976 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.147896    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c467a0e-7b75-45ed-9195-e659a94a676b-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-tkcrj\" (UID: \"8c467a0e-7b75-45ed-9195-e659a94a676b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-tkcrj"
Sep 11 11:35:14.147976 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.147965    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf9x5\" (UniqueName: \"kubernetes.io/projected/2561a637-628e-41ca-a07c-18ba49d56cb2-kube-api-access-bf9x5\") pod \"dns-default-k6szw\" (UID: \"2561a637-628e-41ca-a07c-18ba49d56cb2\") " pod="openshift-dns/dns-default-k6szw"
Sep 11 11:35:14.148157 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.148023    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2561a637-628e-41ca-a07c-18ba49d56cb2-tmp-dir\") pod \"dns-default-k6szw\" (UID: \"2561a637-628e-41ca-a07c-18ba49d56cb2\") " pod="openshift-dns/dns-default-k6szw"
Sep 11 11:35:14.148157 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.148047    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2561a637-628e-41ca-a07c-18ba49d56cb2-config-volume\") pod \"dns-default-k6szw\" (UID: \"2561a637-628e-41ca-a07c-18ba49d56cb2\") " pod="openshift-dns/dns-default-k6szw"
Sep 11 11:35:14.148157 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.148078    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2561a637-628e-41ca-a07c-18ba49d56cb2-metrics-tls\") pod \"dns-default-k6szw\" (UID: \"2561a637-628e-41ca-a07c-18ba49d56cb2\") " pod="openshift-dns/dns-default-k6szw"
Sep 11 11:35:14.148157 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.148106    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c467a0e-7b75-45ed-9195-e659a94a676b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-tkcrj\" (UID: \"8c467a0e-7b75-45ed-9195-e659a94a676b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-tkcrj"
Sep 11 11:35:14.148157 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.148132    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4gcg\" (UniqueName: \"kubernetes.io/projected/8c467a0e-7b75-45ed-9195-e659a94a676b-kube-api-access-g4gcg\") pod \"prometheus-operator-68bcd7f987-tkcrj\" (UID: \"8c467a0e-7b75-45ed-9195-e659a94a676b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-tkcrj"
Sep 11 11:35:14.148358 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.148178    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c467a0e-7b75-45ed-9195-e659a94a676b-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-tkcrj\" (UID: \"8c467a0e-7b75-45ed-9195-e659a94a676b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-tkcrj"
Sep 11 11:35:14.148392 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.148379    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2561a637-628e-41ca-a07c-18ba49d56cb2-tmp-dir\") pod \"dns-default-k6szw\" (UID: \"2561a637-628e-41ca-a07c-18ba49d56cb2\") " pod="openshift-dns/dns-default-k6szw"
Sep 11 11:35:14.148612 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.148594    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2561a637-628e-41ca-a07c-18ba49d56cb2-config-volume\") pod \"dns-default-k6szw\" (UID: \"2561a637-628e-41ca-a07c-18ba49d56cb2\") " pod="openshift-dns/dns-default-k6szw"
Sep 11 11:35:14.150546 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.150524    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2561a637-628e-41ca-a07c-18ba49d56cb2-metrics-tls\") pod \"dns-default-k6szw\" (UID: \"2561a637-628e-41ca-a07c-18ba49d56cb2\") " pod="openshift-dns/dns-default-k6szw"
Sep 11 11:35:14.156821 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.156799    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf9x5\" (UniqueName: \"kubernetes.io/projected/2561a637-628e-41ca-a07c-18ba49d56cb2-kube-api-access-bf9x5\") pod \"dns-default-k6szw\" (UID: \"2561a637-628e-41ca-a07c-18ba49d56cb2\") " pod="openshift-dns/dns-default-k6szw"
Sep 11 11:35:14.157097 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.157072    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:14.224283 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.224261    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wvb2j"
Sep 11 11:35:14.229523 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.229502    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ngm7d"
Sep 11 11:35:14.248657 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.248602    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c467a0e-7b75-45ed-9195-e659a94a676b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-tkcrj\" (UID: \"8c467a0e-7b75-45ed-9195-e659a94a676b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-tkcrj"
Sep 11 11:35:14.248962 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.248680    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4gcg\" (UniqueName: \"kubernetes.io/projected/8c467a0e-7b75-45ed-9195-e659a94a676b-kube-api-access-g4gcg\") pod \"prometheus-operator-68bcd7f987-tkcrj\" (UID: \"8c467a0e-7b75-45ed-9195-e659a94a676b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-tkcrj"
Sep 11 11:35:14.248962 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.248731    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c467a0e-7b75-45ed-9195-e659a94a676b-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-tkcrj\" (UID: \"8c467a0e-7b75-45ed-9195-e659a94a676b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-tkcrj"
Sep 11 11:35:14.248962 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.248757    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c467a0e-7b75-45ed-9195-e659a94a676b-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-tkcrj\" (UID: \"8c467a0e-7b75-45ed-9195-e659a94a676b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-tkcrj"
Sep 11 11:35:14.248962 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:14.248866    2575 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 11 11:35:14.248962 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:14.248928    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c467a0e-7b75-45ed-9195-e659a94a676b-prometheus-operator-tls podName:8c467a0e-7b75-45ed-9195-e659a94a676b nodeName:}" failed. No retries permitted until 2026-09-11 11:35:14.748913057 +0000 UTC m=+55.169519097 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/8c467a0e-7b75-45ed-9195-e659a94a676b-prometheus-operator-tls") pod "prometheus-operator-68bcd7f987-tkcrj" (UID: "8c467a0e-7b75-45ed-9195-e659a94a676b") : secret "prometheus-operator-tls" not found
Sep 11 11:35:14.249453 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.249417    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c467a0e-7b75-45ed-9195-e659a94a676b-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-tkcrj\" (UID: \"8c467a0e-7b75-45ed-9195-e659a94a676b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-tkcrj"
Sep 11 11:35:14.251258 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.251236    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c467a0e-7b75-45ed-9195-e659a94a676b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-tkcrj\" (UID: \"8c467a0e-7b75-45ed-9195-e659a94a676b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-tkcrj"
Sep 11 11:35:14.257287 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.257255    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:14.259770 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.259749    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4gcg\" (UniqueName: \"kubernetes.io/projected/8c467a0e-7b75-45ed-9195-e659a94a676b-kube-api-access-g4gcg\") pod \"prometheus-operator-68bcd7f987-tkcrj\" (UID: \"8c467a0e-7b75-45ed-9195-e659a94a676b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-tkcrj"
Sep 11 11:35:14.322343 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.322313    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-k6szw"
Sep 11 11:35:14.370431 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.370244    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c7c544647-2qvfm"]
Sep 11 11:35:14.374145 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.372786    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:14.375859 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.375454    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 11 11:35:14.375859 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.375753    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 11 11:35:14.375859 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.375766    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 11 11:35:14.376054 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.375909    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 11 11:35:14.376054 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.376020    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 11:35:14.376054 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.376021    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hxjn7\""
Sep 11 11:35:14.376181 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.376108    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 11 11:35:14.376345 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.376276    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 11:35:14.385244 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.385165    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c7c544647-2qvfm"]
Sep 11 11:35:14.451225 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.451146    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/680cef74-e325-4d0a-b6a4-366cdd8f6a58-oauth-serving-cert\") pod \"console-5c7c544647-2qvfm\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") " pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:14.451225 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.451194    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/680cef74-e325-4d0a-b6a4-366cdd8f6a58-console-oauth-config\") pod \"console-5c7c544647-2qvfm\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") " pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:14.451225 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.451219    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62vtd\" (UniqueName: \"kubernetes.io/projected/680cef74-e325-4d0a-b6a4-366cdd8f6a58-kube-api-access-62vtd\") pod \"console-5c7c544647-2qvfm\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") " pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:14.451457 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.451242    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/680cef74-e325-4d0a-b6a4-366cdd8f6a58-console-config\") pod \"console-5c7c544647-2qvfm\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") " pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:14.451457 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.451262    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/680cef74-e325-4d0a-b6a4-366cdd8f6a58-service-ca\") pod \"console-5c7c544647-2qvfm\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") " pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:14.451457 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.451325    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/680cef74-e325-4d0a-b6a4-366cdd8f6a58-console-serving-cert\") pod \"console-5c7c544647-2qvfm\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") " pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:14.524435 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.524408    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ngm7d"]
Sep 11 11:35:14.525563 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:35:14.525532    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29f9b723_dc90_4bf0_bfcb_42bff332e469.slice/crio-f9a289765a94285559e02b7c0ca4db0dbc329ef11fac5abbf76e94a150802d14 WatchSource:0}: Error finding container f9a289765a94285559e02b7c0ca4db0dbc329ef11fac5abbf76e94a150802d14: Status 404 returned error can't find the container with id f9a289765a94285559e02b7c0ca4db0dbc329ef11fac5abbf76e94a150802d14
Sep 11 11:35:14.525974 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:35:14.525947    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd070541c_7f68_4350_a203_048ec6f3cf69.slice/crio-2853894fc94bbdb9ddb23481579b45fb1485a1659b3a0e6fad310015163d64e5 WatchSource:0}: Error finding container 2853894fc94bbdb9ddb23481579b45fb1485a1659b3a0e6fad310015163d64e5: Status 404 returned error can't find the container with id 2853894fc94bbdb9ddb23481579b45fb1485a1659b3a0e6fad310015163d64e5
Sep 11 11:35:14.526746 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.526726    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wvb2j"]
Sep 11 11:35:14.552422 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.552398    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/680cef74-e325-4d0a-b6a4-366cdd8f6a58-console-serving-cert\") pod \"console-5c7c544647-2qvfm\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") " pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:14.552534 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.552459    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/680cef74-e325-4d0a-b6a4-366cdd8f6a58-oauth-serving-cert\") pod \"console-5c7c544647-2qvfm\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") " pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:14.552534 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.552501    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/680cef74-e325-4d0a-b6a4-366cdd8f6a58-console-oauth-config\") pod \"console-5c7c544647-2qvfm\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") " pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:14.552534 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.552522    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62vtd\" (UniqueName: \"kubernetes.io/projected/680cef74-e325-4d0a-b6a4-366cdd8f6a58-kube-api-access-62vtd\") pod \"console-5c7c544647-2qvfm\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") " pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:14.552678 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.552545    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/680cef74-e325-4d0a-b6a4-366cdd8f6a58-console-config\") pod \"console-5c7c544647-2qvfm\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") " pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:14.552678 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.552566    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/680cef74-e325-4d0a-b6a4-366cdd8f6a58-service-ca\") pod \"console-5c7c544647-2qvfm\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") " pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:14.553199 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.553178    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/680cef74-e325-4d0a-b6a4-366cdd8f6a58-console-config\") pod \"console-5c7c544647-2qvfm\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") " pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:14.553296 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.553257    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/680cef74-e325-4d0a-b6a4-366cdd8f6a58-service-ca\") pod \"console-5c7c544647-2qvfm\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") " pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:14.553491 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.553471    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/680cef74-e325-4d0a-b6a4-366cdd8f6a58-oauth-serving-cert\") pod \"console-5c7c544647-2qvfm\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") " pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:14.554944 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.554926    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/680cef74-e325-4d0a-b6a4-366cdd8f6a58-console-serving-cert\") pod \"console-5c7c544647-2qvfm\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") " pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:14.554994 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.554936    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/680cef74-e325-4d0a-b6a4-366cdd8f6a58-console-oauth-config\") pod \"console-5c7c544647-2qvfm\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") " pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:14.561375 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.561356    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62vtd\" (UniqueName: \"kubernetes.io/projected/680cef74-e325-4d0a-b6a4-366cdd8f6a58-kube-api-access-62vtd\") pod \"console-5c7c544647-2qvfm\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") " pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:14.598201 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.598179    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-k6szw"]
Sep 11 11:35:14.601316 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:35:14.601292    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2561a637_628e_41ca_a07c_18ba49d56cb2.slice/crio-f9c2f87a2693b9db4b85396af747707f3a06738fa3f5d5cf96072df5f7bc5f9a WatchSource:0}: Error finding container f9c2f87a2693b9db4b85396af747707f3a06738fa3f5d5cf96072df5f7bc5f9a: Status 404 returned error can't find the container with id f9c2f87a2693b9db4b85396af747707f3a06738fa3f5d5cf96072df5f7bc5f9a
Sep 11 11:35:14.687040 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.687006    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:14.747079 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.747033    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:14.754882 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.754858    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c467a0e-7b75-45ed-9195-e659a94a676b-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-tkcrj\" (UID: \"8c467a0e-7b75-45ed-9195-e659a94a676b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-tkcrj"
Sep 11 11:35:14.757188 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.757163    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c467a0e-7b75-45ed-9195-e659a94a676b-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-tkcrj\" (UID: \"8c467a0e-7b75-45ed-9195-e659a94a676b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-tkcrj"
Sep 11 11:35:14.954097 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.954067    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c7c544647-2qvfm"]
Sep 11 11:35:14.957853 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:35:14.957824    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod680cef74_e325_4d0a_b6a4_366cdd8f6a58.slice/crio-f7ae265f4dc2e04ea54d76ccf17803cb86a8ef802d0b7e66e043ebf4ab35a10b WatchSource:0}: Error finding container f7ae265f4dc2e04ea54d76ccf17803cb86a8ef802d0b7e66e043ebf4ab35a10b: Status 404 returned error can't find the container with id f7ae265f4dc2e04ea54d76ccf17803cb86a8ef802d0b7e66e043ebf4ab35a10b
Sep 11 11:35:14.974181 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:14.974159    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-tkcrj"
Sep 11 11:35:15.076092 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:15.076050    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:35:15.076509 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:15.076492    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:35:15.076787 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:15.076771    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:35:15.079781 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:15.079756    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b45p4\""
Sep 11 11:35:15.080017 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:15.080001    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 11:35:15.080265 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:15.080247    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 11:35:15.080619 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:15.080554    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 11:35:15.080890 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:15.080858    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 11:35:15.081209 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:15.081193    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tk98p\""
Sep 11 11:35:15.250702 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:15.250674    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-tkcrj"]
Sep 11 11:35:15.254343 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:35:15.254312    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c467a0e_7b75_45ed_9195_e659a94a676b.slice/crio-299f576e92fb77f1bb9f5ec9f73cbada4d90ee1be9c4d8e46cc7b0b72e95a38c WatchSource:0}: Error finding container 299f576e92fb77f1bb9f5ec9f73cbada4d90ee1be9c4d8e46cc7b0b72e95a38c: Status 404 returned error can't find the container with id 299f576e92fb77f1bb9f5ec9f73cbada4d90ee1be9c4d8e46cc7b0b72e95a38c
Sep 11 11:35:15.307835 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:15.307804    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-tkcrj" event={"ID":"8c467a0e-7b75-45ed-9195-e659a94a676b","Type":"ContainerStarted","Data":"299f576e92fb77f1bb9f5ec9f73cbada4d90ee1be9c4d8e46cc7b0b72e95a38c"}
Sep 11 11:35:15.309821 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:15.309745    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ngm7d" event={"ID":"d070541c-7f68-4350-a203-048ec6f3cf69","Type":"ContainerStarted","Data":"f01305da011c2ec98c228f775aa0d48a7df6b3031d2f464cf0a6d07c6eeb0eff"}
Sep 11 11:35:15.309821 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:15.309795    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ngm7d" event={"ID":"d070541c-7f68-4350-a203-048ec6f3cf69","Type":"ContainerStarted","Data":"2853894fc94bbdb9ddb23481579b45fb1485a1659b3a0e6fad310015163d64e5"}
Sep 11 11:35:15.312255 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:15.312226    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wvb2j" event={"ID":"29f9b723-dc90-4bf0-bfcb-42bff332e469","Type":"ContainerStarted","Data":"f9a289765a94285559e02b7c0ca4db0dbc329ef11fac5abbf76e94a150802d14"}
Sep 11 11:35:15.314919 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:15.314845    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c7c544647-2qvfm" event={"ID":"680cef74-e325-4d0a-b6a4-366cdd8f6a58","Type":"ContainerStarted","Data":"f7ae265f4dc2e04ea54d76ccf17803cb86a8ef802d0b7e66e043ebf4ab35a10b"}
Sep 11 11:35:15.316684 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:15.316659    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k6szw" event={"ID":"2561a637-628e-41ca-a07c-18ba49d56cb2","Type":"ContainerStarted","Data":"f9c2f87a2693b9db4b85396af747707f3a06738fa3f5d5cf96072df5f7bc5f9a"}
Sep 11 11:35:15.745850 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:15.745814    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:15.849212 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:15.849126    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:15.947055 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:15.947010    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:16.323046 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:16.322960    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ngm7d" event={"ID":"d070541c-7f68-4350-a203-048ec6f3cf69","Type":"ContainerStarted","Data":"9448eed04359d64efcd90c36c86b4a8e53e6b770b2117fd0df8c4769b9bb1842"}
Sep 11 11:35:17.347399 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:17.347361    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:17.447733 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:17.447678    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:18.412957 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:18.412916    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:18.468799 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:18.468765    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:18.971936 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:18.971857    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:19.073110 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:19.073075    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:19.962922 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:19.962830    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:20.256743 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:20.256712    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rzhmb"
Sep 11 11:35:20.337675 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:20.337621    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c7c544647-2qvfm" event={"ID":"680cef74-e325-4d0a-b6a4-366cdd8f6a58","Type":"ContainerStarted","Data":"9d978a5a48996e6f00a1bfe079793608c7bb5b8e9e0031cf1a249770fbed3185"}
Sep 11 11:35:20.339408 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:20.339380    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k6szw" event={"ID":"2561a637-628e-41ca-a07c-18ba49d56cb2","Type":"ContainerStarted","Data":"7da683e9fb6a0d2b49e3e4bef94aab2d7f9d4828a4188b783bed985353ed3f07"}
Sep 11 11:35:20.339554 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:20.339428    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k6szw" event={"ID":"2561a637-628e-41ca-a07c-18ba49d56cb2","Type":"ContainerStarted","Data":"709d125b7a9a5c66db21fee3a51512ecd73f88e8dbcbcecf176144b1f901ce8c"}
Sep 11 11:35:20.339554 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:20.339453    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-k6szw"
Sep 11 11:35:20.341122 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:20.341092    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-tkcrj" event={"ID":"8c467a0e-7b75-45ed-9195-e659a94a676b","Type":"ContainerStarted","Data":"07047b872c18e3153c30883d56ecca32333e3659a4890df4a109757f1e995209"}
Sep 11 11:35:20.341245 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:20.341134    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-tkcrj" event={"ID":"8c467a0e-7b75-45ed-9195-e659a94a676b","Type":"ContainerStarted","Data":"9999b48657da444ecc0c7b9a39a781c1feb3d82da4398f88605fe435dbe0ff87"}
Sep 11 11:35:20.343085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:20.343068    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ngm7d" event={"ID":"d070541c-7f68-4350-a203-048ec6f3cf69","Type":"ContainerStarted","Data":"439fc047f8acdd9f9eb2e31562a61ed1bf3a50548372d2b733bac3eea2211195"}
Sep 11 11:35:20.344532 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:20.344506    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wvb2j" event={"ID":"29f9b723-dc90-4bf0-bfcb-42bff332e469","Type":"ContainerStarted","Data":"9c4625d3e5ae5d40e2bbde43a3b033d6c78c705a1981fe0472c25768f32160c3"}
Sep 11 11:35:20.355899 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:20.355858    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c7c544647-2qvfm" podStartSLOduration=1.734438286 podStartE2EDuration="6.355845097s" totalImagesPullingTime="4.621406811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:35:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:35:20.355383008 +0000 UTC m=+60.775989039" watchObservedRunningTime="2026-09-11 11:35:20.355845097 +0000 UTC m=+60.776451116"
Sep 11 11:35:20.371527 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:20.371480    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wvb2j" podStartSLOduration=2.323725406 podStartE2EDuration="7.371468884s" totalImagesPullingTime="5.047743478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:35:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:35:20.37107784 +0000 UTC m=+60.791683867" watchObservedRunningTime="2026-09-11 11:35:20.371468884 +0000 UTC m=+60.792074912"
Sep 11 11:35:20.384776 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:20.384752    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:20.390173 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:20.390126    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ngm7d" podStartSLOduration=2.501871274 podStartE2EDuration="7.390108071s" totalImagesPullingTime="4.888236797s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:35:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:35:20.389042959 +0000 UTC m=+60.809648986" watchObservedRunningTime="2026-09-11 11:35:20.390108071 +0000 UTC m=+60.810714164"
Sep 11 11:35:20.410794 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:20.410760    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-k6szw" podStartSLOduration=2.438563209 podStartE2EDuration="7.410746347s" totalImagesPullingTime="4.972183138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:35:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:35:20.41061491 +0000 UTC m=+60.831220939" watchObservedRunningTime="2026-09-11 11:35:20.410746347 +0000 UTC m=+60.831352378"
Sep 11 11:35:20.432646 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:20.432592    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-tkcrj" podStartSLOduration=2.116688172 podStartE2EDuration="6.432579566s" totalImagesPullingTime="4.315891394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:35:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:35:20.432328101 +0000 UTC m=+60.852934129" watchObservedRunningTime="2026-09-11 11:35:20.432579566 +0000 UTC m=+60.853185593"
Sep 11 11:35:20.461011 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:20.460973    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:21.939167 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:21.939135    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-65f55d9874-p6xzw"]
Sep 11 11:35:21.941848 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:21.941833    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:21.949743 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:21.949722    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 11 11:35:21.954535 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:21.954515    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65f55d9874-p6xzw"]
Sep 11 11:35:22.016372 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.016336    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-trusted-ca-bundle\") pod \"console-65f55d9874-p6xzw\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") " pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:22.016549 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.016384    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d08cc13-d976-4959-8fc7-a851d6d4c82d-console-serving-cert\") pod \"console-65f55d9874-p6xzw\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") " pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:22.016549 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.016442    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d08cc13-d976-4959-8fc7-a851d6d4c82d-console-oauth-config\") pod \"console-65f55d9874-p6xzw\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") " pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:22.016549 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.016462    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-console-config\") pod \"console-65f55d9874-p6xzw\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") " pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:22.016549 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.016493    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-oauth-serving-cert\") pod \"console-65f55d9874-p6xzw\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") " pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:22.016713 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.016554    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-service-ca\") pod \"console-65f55d9874-p6xzw\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") " pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:22.016713 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.016570    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bdhk\" (UniqueName: \"kubernetes.io/projected/5d08cc13-d976-4959-8fc7-a851d6d4c82d-kube-api-access-5bdhk\") pod \"console-65f55d9874-p6xzw\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") " pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:22.117340 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.117305    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d08cc13-d976-4959-8fc7-a851d6d4c82d-console-oauth-config\") pod \"console-65f55d9874-p6xzw\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") " pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:22.117496 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.117357    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-console-config\") pod \"console-65f55d9874-p6xzw\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") " pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:22.117496 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.117386    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-oauth-serving-cert\") pod \"console-65f55d9874-p6xzw\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") " pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:22.117496 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.117431    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-service-ca\") pod \"console-65f55d9874-p6xzw\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") " pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:22.117496 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.117470    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bdhk\" (UniqueName: \"kubernetes.io/projected/5d08cc13-d976-4959-8fc7-a851d6d4c82d-kube-api-access-5bdhk\") pod \"console-65f55d9874-p6xzw\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") " pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:22.117723 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.117501    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-trusted-ca-bundle\") pod \"console-65f55d9874-p6xzw\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") " pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:22.117723 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.117544    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d08cc13-d976-4959-8fc7-a851d6d4c82d-console-serving-cert\") pod \"console-65f55d9874-p6xzw\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") " pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:22.118228 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.118203    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-service-ca\") pod \"console-65f55d9874-p6xzw\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") " pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:22.118327 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.118229    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-oauth-serving-cert\") pod \"console-65f55d9874-p6xzw\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") " pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:22.118327 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.118229    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-console-config\") pod \"console-65f55d9874-p6xzw\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") " pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:22.118396 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.118358    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-trusted-ca-bundle\") pod \"console-65f55d9874-p6xzw\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") " pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:22.120451 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.120435    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d08cc13-d976-4959-8fc7-a851d6d4c82d-console-oauth-config\") pod \"console-65f55d9874-p6xzw\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") " pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:22.120686 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.120667    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d08cc13-d976-4959-8fc7-a851d6d4c82d-console-serving-cert\") pod \"console-65f55d9874-p6xzw\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") " pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:22.128505 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.128486    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bdhk\" (UniqueName: \"kubernetes.io/projected/5d08cc13-d976-4959-8fc7-a851d6d4c82d-kube-api-access-5bdhk\") pod \"console-65f55d9874-p6xzw\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") " pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:22.249952 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.249925    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:22.250395 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.250360    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:22.382116 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.382079    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:22.545080 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.545058    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65f55d9874-p6xzw"]
Sep 11 11:35:22.545896 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:35:22.545873    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d08cc13_d976_4959_8fc7_a851d6d4c82d.slice/crio-efd3ea5d4adffd359d8eb93b0def9c6f1b335c792903c36a5ecb67a98b90e1b4 WatchSource:0}: Error finding container efd3ea5d4adffd359d8eb93b0def9c6f1b335c792903c36a5ecb67a98b90e1b4: Status 404 returned error can't find the container with id efd3ea5d4adffd359d8eb93b0def9c6f1b335c792903c36a5ecb67a98b90e1b4
Sep 11 11:35:22.621490 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.621457    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:22.676076 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.676049    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-wvflm"]
Sep 11 11:35:22.678164 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.678149    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wvflm"
Sep 11 11:35:22.682130 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.682105    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 11 11:35:22.682246 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.682150    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-kt2cq\""
Sep 11 11:35:22.682246 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.682118    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 11 11:35:22.710428 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.710400    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-wvflm"]
Sep 11 11:35:22.711779 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.711758    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-hz459"]
Sep 11 11:35:22.713877 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.713861    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459"
Sep 11 11:35:22.721363 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.721344    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/750518f9-62fd-4788-a9d4-2794dbcdc844-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-wvflm\" (UID: \"750518f9-62fd-4788-a9d4-2794dbcdc844\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wvflm"
Sep 11 11:35:22.721454 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.721372    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmvvf\" (UniqueName: \"kubernetes.io/projected/750518f9-62fd-4788-a9d4-2794dbcdc844-kube-api-access-fmvvf\") pod \"openshift-state-metrics-67dcf8c979-wvflm\" (UID: \"750518f9-62fd-4788-a9d4-2794dbcdc844\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wvflm"
Sep 11 11:35:22.721454 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.721392    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/750518f9-62fd-4788-a9d4-2794dbcdc844-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-wvflm\" (UID: \"750518f9-62fd-4788-a9d4-2794dbcdc844\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wvflm"
Sep 11 11:35:22.721535 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.721454    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/750518f9-62fd-4788-a9d4-2794dbcdc844-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-wvflm\" (UID: \"750518f9-62fd-4788-a9d4-2794dbcdc844\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wvflm"
Sep 11 11:35:22.726504 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.726488    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 11 11:35:22.726714 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.726699    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-xpqxh\""
Sep 11 11:35:22.726948 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.726934    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 11 11:35:22.727746 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.727732    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 11 11:35:22.739163 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.739143    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-hz459"]
Sep 11 11:35:22.739991 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.739970    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-r8742"]
Sep 11 11:35:22.742028 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.742015    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.745496 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.745477    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 11:35:22.745699 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.745683    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fcpz2\""
Sep 11 11:35:22.745787 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.745749    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 11:35:22.746651 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.746621    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 11:35:22.771750 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.771723    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:22.822555 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.822483    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.822555 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.822514    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/750518f9-62fd-4788-a9d4-2794dbcdc844-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-wvflm\" (UID: \"750518f9-62fd-4788-a9d4-2794dbcdc844\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wvflm"
Sep 11 11:35:22.822555 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.822532    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-node-exporter-wtmp\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.822555 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.822548    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmvvf\" (UniqueName: \"kubernetes.io/projected/750518f9-62fd-4788-a9d4-2794dbcdc844-kube-api-access-fmvvf\") pod \"openshift-state-metrics-67dcf8c979-wvflm\" (UID: \"750518f9-62fd-4788-a9d4-2794dbcdc844\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wvflm"
Sep 11 11:35:22.822828 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.822567    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/750518f9-62fd-4788-a9d4-2794dbcdc844-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-wvflm\" (UID: \"750518f9-62fd-4788-a9d4-2794dbcdc844\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wvflm"
Sep 11 11:35:22.822828 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.822591    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-node-exporter-tls\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.822828 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.822733    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/840bc49f-a645-4e7b-a924-0a26013e9882-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-hz459\" (UID: \"840bc49f-a645-4e7b-a924-0a26013e9882\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459"
Sep 11 11:35:22.822828 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.822771    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/750518f9-62fd-4788-a9d4-2794dbcdc844-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-wvflm\" (UID: \"750518f9-62fd-4788-a9d4-2794dbcdc844\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wvflm"
Sep 11 11:35:22.822828 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.822793    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdx49\" (UniqueName: \"kubernetes.io/projected/840bc49f-a645-4e7b-a924-0a26013e9882-kube-api-access-cdx49\") pod \"kube-state-metrics-59fbdd77d-hz459\" (UID: \"840bc49f-a645-4e7b-a924-0a26013e9882\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459"
Sep 11 11:35:22.822828 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.822812    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-sys\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.823043 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.822883    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/840bc49f-a645-4e7b-a924-0a26013e9882-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-hz459\" (UID: \"840bc49f-a645-4e7b-a924-0a26013e9882\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459"
Sep 11 11:35:22.823043 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.822925    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/840bc49f-a645-4e7b-a924-0a26013e9882-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-hz459\" (UID: \"840bc49f-a645-4e7b-a924-0a26013e9882\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459"
Sep 11 11:35:22.823043 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.822953    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-root\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.823043 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.822977    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-node-exporter-accelerators-collector-config\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.823043 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.823002    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnxxc\" (UniqueName: \"kubernetes.io/projected/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-kube-api-access-pnxxc\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.823264 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.823053    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-node-exporter-textfile\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.823264 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.823077    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/840bc49f-a645-4e7b-a924-0a26013e9882-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-hz459\" (UID: \"840bc49f-a645-4e7b-a924-0a26013e9882\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459"
Sep 11 11:35:22.823264 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.823118    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-metrics-client-ca\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.823264 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.823161    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/840bc49f-a645-4e7b-a924-0a26013e9882-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-hz459\" (UID: \"840bc49f-a645-4e7b-a924-0a26013e9882\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459"
Sep 11 11:35:22.823427 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.823409    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/750518f9-62fd-4788-a9d4-2794dbcdc844-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-wvflm\" (UID: \"750518f9-62fd-4788-a9d4-2794dbcdc844\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wvflm"
Sep 11 11:35:22.825102 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.825085    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/750518f9-62fd-4788-a9d4-2794dbcdc844-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-wvflm\" (UID: \"750518f9-62fd-4788-a9d4-2794dbcdc844\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wvflm"
Sep 11 11:35:22.825177 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.825162    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/750518f9-62fd-4788-a9d4-2794dbcdc844-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-wvflm\" (UID: \"750518f9-62fd-4788-a9d4-2794dbcdc844\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wvflm"
Sep 11 11:35:22.846602 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.846571    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmvvf\" (UniqueName: \"kubernetes.io/projected/750518f9-62fd-4788-a9d4-2794dbcdc844-kube-api-access-fmvvf\") pod \"openshift-state-metrics-67dcf8c979-wvflm\" (UID: \"750518f9-62fd-4788-a9d4-2794dbcdc844\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wvflm"
Sep 11 11:35:22.896052 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.896018    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:22.924353 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.924331    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.924468 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.924359    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-node-exporter-wtmp\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.924468 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.924381    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-node-exporter-tls\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.924468 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.924397    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/840bc49f-a645-4e7b-a924-0a26013e9882-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-hz459\" (UID: \"840bc49f-a645-4e7b-a924-0a26013e9882\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459"
Sep 11 11:35:22.924468 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.924414    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdx49\" (UniqueName: \"kubernetes.io/projected/840bc49f-a645-4e7b-a924-0a26013e9882-kube-api-access-cdx49\") pod \"kube-state-metrics-59fbdd77d-hz459\" (UID: \"840bc49f-a645-4e7b-a924-0a26013e9882\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459"
Sep 11 11:35:22.924468 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.924428    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-sys\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.924468 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.924466    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/840bc49f-a645-4e7b-a924-0a26013e9882-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-hz459\" (UID: \"840bc49f-a645-4e7b-a924-0a26013e9882\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459"
Sep 11 11:35:22.925416 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.924488    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/840bc49f-a645-4e7b-a924-0a26013e9882-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-hz459\" (UID: \"840bc49f-a645-4e7b-a924-0a26013e9882\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459"
Sep 11 11:35:22.925416 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.924506    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-root\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.925416 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.924520    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-sys\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.925416 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.924531    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-node-exporter-accelerators-collector-config\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.925416 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.924558    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnxxc\" (UniqueName: \"kubernetes.io/projected/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-kube-api-access-pnxxc\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.925416 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.924570    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-root\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.925416 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.924532    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-node-exporter-wtmp\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.925416 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:22.924710    2575 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 11 11:35:22.925416 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.924754    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-node-exporter-textfile\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.925416 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:22.924792    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-node-exporter-tls podName:271013ff-ead3-4ebc-aa2e-cfc82bede0d7 nodeName:}" failed. No retries permitted until 2026-09-11 11:35:23.424771309 +0000 UTC m=+63.845377321 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-node-exporter-tls") pod "node-exporter-r8742" (UID: "271013ff-ead3-4ebc-aa2e-cfc82bede0d7") : secret "node-exporter-tls" not found
Sep 11 11:35:22.925416 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.924826    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/840bc49f-a645-4e7b-a924-0a26013e9882-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-hz459\" (UID: \"840bc49f-a645-4e7b-a924-0a26013e9882\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459"
Sep 11 11:35:22.925416 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.924854    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-metrics-client-ca\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.925416 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.924892    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/840bc49f-a645-4e7b-a924-0a26013e9882-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-hz459\" (UID: \"840bc49f-a645-4e7b-a924-0a26013e9882\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459"
Sep 11 11:35:22.925416 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.924962    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/840bc49f-a645-4e7b-a924-0a26013e9882-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-hz459\" (UID: \"840bc49f-a645-4e7b-a924-0a26013e9882\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459"
Sep 11 11:35:22.925416 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.925059    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-node-exporter-textfile\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.925416 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.925188    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/840bc49f-a645-4e7b-a924-0a26013e9882-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-hz459\" (UID: \"840bc49f-a645-4e7b-a924-0a26013e9882\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459"
Sep 11 11:35:22.925989 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.925205    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-node-exporter-accelerators-collector-config\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.925989 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.925584    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/840bc49f-a645-4e7b-a924-0a26013e9882-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-hz459\" (UID: \"840bc49f-a645-4e7b-a924-0a26013e9882\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459"
Sep 11 11:35:22.925989 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.925684    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-metrics-client-ca\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.927144 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.927127    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.927263 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.927247    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/840bc49f-a645-4e7b-a924-0a26013e9882-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-hz459\" (UID: \"840bc49f-a645-4e7b-a924-0a26013e9882\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459"
Sep 11 11:35:22.927309 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.927293    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/840bc49f-a645-4e7b-a924-0a26013e9882-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-hz459\" (UID: \"840bc49f-a645-4e7b-a924-0a26013e9882\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459"
Sep 11 11:35:22.941038 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.941012    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdx49\" (UniqueName: \"kubernetes.io/projected/840bc49f-a645-4e7b-a924-0a26013e9882-kube-api-access-cdx49\") pod \"kube-state-metrics-59fbdd77d-hz459\" (UID: \"840bc49f-a645-4e7b-a924-0a26013e9882\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459"
Sep 11 11:35:22.942913 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.942897    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnxxc\" (UniqueName: \"kubernetes.io/projected/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-kube-api-access-pnxxc\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:22.989926 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:22.989905    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wvflm"
Sep 11 11:35:23.022316 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.022290    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459"
Sep 11 11:35:23.120242 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.120203    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:23.234211 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.234182    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:23.300957 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.300926    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-wvflm"]
Sep 11 11:35:23.304767 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:35:23.304743    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod750518f9_62fd_4788_a9d4_2794dbcdc844.slice/crio-968a11387d03f541e239b54b593dc4ccd7b6b42bf50c5838d46d547074d2861e WatchSource:0}: Error finding container 968a11387d03f541e239b54b593dc4ccd7b6b42bf50c5838d46d547074d2861e: Status 404 returned error can't find the container with id 968a11387d03f541e239b54b593dc4ccd7b6b42bf50c5838d46d547074d2861e
Sep 11 11:35:23.332777 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.332724    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-hz459"]
Sep 11 11:35:23.335715 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:35:23.335685    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod840bc49f_a645_4e7b_a924_0a26013e9882.slice/crio-f91e97f3845a3328cacd88394ec419511aa0901ba6292a8d73845b964f3ec81d WatchSource:0}: Error finding container f91e97f3845a3328cacd88394ec419511aa0901ba6292a8d73845b964f3ec81d: Status 404 returned error can't find the container with id f91e97f3845a3328cacd88394ec419511aa0901ba6292a8d73845b964f3ec81d
Sep 11 11:35:23.355556 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.355532    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65f55d9874-p6xzw" event={"ID":"5d08cc13-d976-4959-8fc7-a851d6d4c82d","Type":"ContainerStarted","Data":"81bf5aeab9607d8ce5dea533dc0dfb4b471c1541e963e6cd5de9b2869135441d"}
Sep 11 11:35:23.355668 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.355583    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65f55d9874-p6xzw" event={"ID":"5d08cc13-d976-4959-8fc7-a851d6d4c82d","Type":"ContainerStarted","Data":"efd3ea5d4adffd359d8eb93b0def9c6f1b335c792903c36a5ecb67a98b90e1b4"}
Sep 11 11:35:23.356823 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.356799    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459" event={"ID":"840bc49f-a645-4e7b-a924-0a26013e9882","Type":"ContainerStarted","Data":"f91e97f3845a3328cacd88394ec419511aa0901ba6292a8d73845b964f3ec81d"}
Sep 11 11:35:23.357892 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.357871    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wvflm" event={"ID":"750518f9-62fd-4788-a9d4-2794dbcdc844","Type":"ContainerStarted","Data":"968a11387d03f541e239b54b593dc4ccd7b6b42bf50c5838d46d547074d2861e"}
Sep 11 11:35:23.377393 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.377346    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-65f55d9874-p6xzw" podStartSLOduration=2.377326393 podStartE2EDuration="2.377326393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:35:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:35:23.376904241 +0000 UTC m=+63.797510272" watchObservedRunningTime="2026-09-11 11:35:23.377326393 +0000 UTC m=+63.797932421"
Sep 11 11:35:23.430445 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.429510    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-node-exporter-tls\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:23.432382 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.432360    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/271013ff-ead3-4ebc-aa2e-cfc82bede0d7-node-exporter-tls\") pod \"node-exporter-r8742\" (UID: \"271013ff-ead3-4ebc-aa2e-cfc82bede0d7\") " pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:23.477607 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.477569    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:23.567269 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.567232    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:35:23.569845 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.569812    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.572764 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.572470    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 11 11:35:23.572764 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.572470    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 11 11:35:23.572764 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.572470    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 11 11:35:23.572764 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.572612    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 11 11:35:23.574019 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.573990    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:23.574301 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.574281    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 11 11:35:23.574408 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.574313    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 11 11:35:23.574408 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.574269    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 11 11:35:23.574560 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.574542    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-l2r4q\""
Sep 11 11:35:23.574768 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.574755    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 11 11:35:23.574837 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.574766    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 11 11:35:23.589226 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.589167    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:35:23.630288 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.630255    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.630455 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.630296    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bbe42078-01c1-49d6-a4df-785832fb726e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.630455 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.630329    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbe42078-01c1-49d6-a4df-785832fb726e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.630455 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.630371    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bbe42078-01c1-49d6-a4df-785832fb726e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.630455 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.630399    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.630455 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.630420    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.630755 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.630487    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-config-volume\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.630755 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.630511    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.630755 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.630548    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jd9l\" (UniqueName: \"kubernetes.io/projected/bbe42078-01c1-49d6-a4df-785832fb726e-kube-api-access-9jd9l\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.630755 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.630677    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-web-config\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.630755 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.630708    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.630755 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.630753    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbe42078-01c1-49d6-a4df-785832fb726e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.631014 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.630793    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bbe42078-01c1-49d6-a4df-785832fb726e-config-out\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.651262 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.650996    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r8742"
Sep 11 11:35:23.660266 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:35:23.660232    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod271013ff_ead3_4ebc_aa2e_cfc82bede0d7.slice/crio-204fc62c75b4631cca96c3312d9f08f7ed96d963edd24efe71101ee81dd99af2 WatchSource:0}: Error finding container 204fc62c75b4631cca96c3312d9f08f7ed96d963edd24efe71101ee81dd99af2: Status 404 returned error can't find the container with id 204fc62c75b4631cca96c3312d9f08f7ed96d963edd24efe71101ee81dd99af2
Sep 11 11:35:23.731967 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.731918    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.731967 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.731970    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bbe42078-01c1-49d6-a4df-785832fb726e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.732174 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.732095    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbe42078-01c1-49d6-a4df-785832fb726e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.732174 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.732137    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bbe42078-01c1-49d6-a4df-785832fb726e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.732290 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:23.732271    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bbe42078-01c1-49d6-a4df-785832fb726e-alertmanager-trusted-ca-bundle podName:bbe42078-01c1-49d6-a4df-785832fb726e nodeName:}" failed. No retries permitted until 2026-09-11 11:35:24.232246349 +0000 UTC m=+64.652852400 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/bbe42078-01c1-49d6-a4df-785832fb726e-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "bbe42078-01c1-49d6-a4df-785832fb726e") : configmap references non-existent config key: ca-bundle.crt
Sep 11 11:35:23.732355 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.732301    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.732355 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.732328    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.732447 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.732417    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-config-volume\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.732498 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.732441    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bbe42078-01c1-49d6-a4df-785832fb726e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.732498 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.732450    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.732591 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.732510    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jd9l\" (UniqueName: \"kubernetes.io/projected/bbe42078-01c1-49d6-a4df-785832fb726e-kube-api-access-9jd9l\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.732591 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.732544    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-web-config\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.732591 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.732572    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.732787 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.732621    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbe42078-01c1-49d6-a4df-785832fb726e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.732787 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.732685    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bbe42078-01c1-49d6-a4df-785832fb726e-config-out\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.733260 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:23.733235    2575 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 11 11:35:23.733366 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:23.733301    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-main-tls podName:bbe42078-01c1-49d6-a4df-785832fb726e nodeName:}" failed. No retries permitted until 2026-09-11 11:35:24.233281973 +0000 UTC m=+64.653887979 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "bbe42078-01c1-49d6-a4df-785832fb726e") : secret "alertmanager-main-tls" not found
Sep 11 11:35:23.733973 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.733947    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbe42078-01c1-49d6-a4df-785832fb726e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.735421 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.735394    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bbe42078-01c1-49d6-a4df-785832fb726e-config-out\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.735510 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.735432    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.735510 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.735453    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.735626 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.735560    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-config-volume\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.735626 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.735560    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.736047 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.736028    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.736216 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.736196    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-web-config\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.736394 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.736376    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bbe42078-01c1-49d6-a4df-785832fb726e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.743325 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.743305    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jd9l\" (UniqueName: \"kubernetes.io/projected/bbe42078-01c1-49d6-a4df-785832fb726e-kube-api-access-9jd9l\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:23.872993 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:23.872664    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:24.239484 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:24.239447    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:24.239943 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:24.239543    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbe42078-01c1-49d6-a4df-785832fb726e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:24.240514 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:24.240492    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbe42078-01c1-49d6-a4df-785832fb726e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:24.242273 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:24.242249    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:24.364715 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:24.364670    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wvflm" event={"ID":"750518f9-62fd-4788-a9d4-2794dbcdc844","Type":"ContainerStarted","Data":"b3acd6b549127430606b34bbcb3890d57aa857b70d40bd6fd27b695c31031099"}
Sep 11 11:35:24.364872 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:24.364745    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wvflm" event={"ID":"750518f9-62fd-4788-a9d4-2794dbcdc844","Type":"ContainerStarted","Data":"743c2f5bd70c49bef8e3223119d57ad4c6a0233c2f0ba59c4041f5fbdcdc9626"}
Sep 11 11:35:24.365957 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:24.365920    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r8742" event={"ID":"271013ff-ead3-4ebc-aa2e-cfc82bede0d7","Type":"ContainerStarted","Data":"204fc62c75b4631cca96c3312d9f08f7ed96d963edd24efe71101ee81dd99af2"}
Sep 11 11:35:24.484060 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:24.484028    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:35:24.687687 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:24.687581    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:24.687687 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:24.687678    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:24.693590 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:24.693568    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:25.030653 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.030601    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:35:25.034714 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:35:25.034689    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbe42078_01c1_49d6_a4df_785832fb726e.slice/crio-1c8fedaba5e3786609ef657d74b90785c4eeb85e1622261d4bf0505c7ba42c3e WatchSource:0}: Error finding container 1c8fedaba5e3786609ef657d74b90785c4eeb85e1622261d4bf0505c7ba42c3e: Status 404 returned error can't find the container with id 1c8fedaba5e3786609ef657d74b90785c4eeb85e1622261d4bf0505c7ba42c3e
Sep 11 11:35:25.359107 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.359072    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:25.371590 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.371561    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wvflm" event={"ID":"750518f9-62fd-4788-a9d4-2794dbcdc844","Type":"ContainerStarted","Data":"e0a8b2b1e68e78999b073b4a41bef1d254bcea6c5951f58b1942b379b3efb29a"}
Sep 11 11:35:25.377405 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.377354    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459" event={"ID":"840bc49f-a645-4e7b-a924-0a26013e9882","Type":"ContainerStarted","Data":"6b85e47b459ba0ee03f2cd8d43f5996a549fb60553811649c58bfcd093fe552d"}
Sep 11 11:35:25.377550 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.377445    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459" event={"ID":"840bc49f-a645-4e7b-a924-0a26013e9882","Type":"ContainerStarted","Data":"f0a6c98a25059c1ebba8f48180567c4b7b7d9be44c24560ea351822784887b10"}
Sep 11 11:35:25.377550 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.377476    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459" event={"ID":"840bc49f-a645-4e7b-a924-0a26013e9882","Type":"ContainerStarted","Data":"876534100aa24d1d463b7c9e3fbab597416b90cf6c6fcd81ace5d0f0a05cc756"}
Sep 11 11:35:25.378675 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.378627    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bbe42078-01c1-49d6-a4df-785832fb726e","Type":"ContainerStarted","Data":"1c8fedaba5e3786609ef657d74b90785c4eeb85e1622261d4bf0505c7ba42c3e"}
Sep 11 11:35:25.382797 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.382779    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:35:25.401275 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.401229    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-wvflm" podStartSLOduration=2.075600365 podStartE2EDuration="3.40121244s" totalImagesPullingTime="1.325612075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:35:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:35:25.401198715 +0000 UTC m=+65.821804744" watchObservedRunningTime="2026-09-11 11:35:25.40121244 +0000 UTC m=+65.821818470"
Sep 11 11:35:25.454329 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.454285    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hz459" podStartSLOduration=2.032687323 podStartE2EDuration="3.454267206s" totalImagesPullingTime="1.421579883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:35:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:35:25.453906439 +0000 UTC m=+65.874512471" watchObservedRunningTime="2026-09-11 11:35:25.454267206 +0000 UTC m=+65.874873233"
Sep 11 11:35:25.461532 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.461497    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:25.549571 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.549528    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:25.563045 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.563021    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5978f55466-qxmnm"]
Sep 11 11:35:25.567128 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.567110    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.569700 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.569678    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-rpghd\""
Sep 11 11:35:25.569700 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.569704    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 11 11:35:25.569875 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.569723    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 11 11:35:25.569875 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.569756    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 11 11:35:25.569875 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.569704    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 11 11:35:25.569875 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.569707    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 11 11:35:25.570130 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.570115    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-dqaetkjjt28dc\""
Sep 11 11:35:25.582842 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.582823    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5978f55466-qxmnm"]
Sep 11 11:35:25.650002 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.649922    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:25.655005 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.654977    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/45118d7e-8d89-47ed-aa1a-47d58f96bb73-secret-thanos-querier-tls\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.655160 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.655051    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92v5s\" (UniqueName: \"kubernetes.io/projected/45118d7e-8d89-47ed-aa1a-47d58f96bb73-kube-api-access-92v5s\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.655235 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.655181    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/45118d7e-8d89-47ed-aa1a-47d58f96bb73-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.655235 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.655221    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/45118d7e-8d89-47ed-aa1a-47d58f96bb73-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.655342 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.655275    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45118d7e-8d89-47ed-aa1a-47d58f96bb73-metrics-client-ca\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.655342 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.655298    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/45118d7e-8d89-47ed-aa1a-47d58f96bb73-secret-grpc-tls\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.655449 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.655338    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/45118d7e-8d89-47ed-aa1a-47d58f96bb73-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.655449 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.655365    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/45118d7e-8d89-47ed-aa1a-47d58f96bb73-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.756369 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.756336    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/45118d7e-8d89-47ed-aa1a-47d58f96bb73-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.756505 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.756385    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45118d7e-8d89-47ed-aa1a-47d58f96bb73-metrics-client-ca\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.756505 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.756404    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/45118d7e-8d89-47ed-aa1a-47d58f96bb73-secret-grpc-tls\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.756505 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.756441    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/45118d7e-8d89-47ed-aa1a-47d58f96bb73-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.756505 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.756461    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/45118d7e-8d89-47ed-aa1a-47d58f96bb73-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.756505 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.756483    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/45118d7e-8d89-47ed-aa1a-47d58f96bb73-secret-thanos-querier-tls\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.756754 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.756677    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92v5s\" (UniqueName: \"kubernetes.io/projected/45118d7e-8d89-47ed-aa1a-47d58f96bb73-kube-api-access-92v5s\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.756810 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.756784    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs\") pod \"network-metrics-daemon-2tbn7\" (UID: \"7d81bb62-6b31-4915-8827-f8d7347665c0\") " pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:35:25.756859 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.756824    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/45118d7e-8d89-47ed-aa1a-47d58f96bb73-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.757399 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.757364    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45118d7e-8d89-47ed-aa1a-47d58f96bb73-metrics-client-ca\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.759217 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.759198    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/45118d7e-8d89-47ed-aa1a-47d58f96bb73-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.759325 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.759295    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/45118d7e-8d89-47ed-aa1a-47d58f96bb73-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.759775 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.759752    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/45118d7e-8d89-47ed-aa1a-47d58f96bb73-secret-grpc-tls\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.759854 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.759817    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/45118d7e-8d89-47ed-aa1a-47d58f96bb73-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.759954 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.759934    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/45118d7e-8d89-47ed-aa1a-47d58f96bb73-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.759987 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.759941    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/45118d7e-8d89-47ed-aa1a-47d58f96bb73-secret-thanos-querier-tls\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.760333 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.760317    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 11:35:25.766452 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.766431    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92v5s\" (UniqueName: \"kubernetes.io/projected/45118d7e-8d89-47ed-aa1a-47d58f96bb73-kube-api-access-92v5s\") pod \"thanos-querier-5978f55466-qxmnm\" (UID: \"45118d7e-8d89-47ed-aa1a-47d58f96bb73\") " pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.769128 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.769110    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d81bb62-6b31-4915-8827-f8d7347665c0-metrics-certs\") pod \"network-metrics-daemon-2tbn7\" (UID: \"7d81bb62-6b31-4915-8827-f8d7347665c0\") " pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:35:25.858176 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.858134    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret\") pod \"global-pull-secret-syncer-nhrgs\" (UID: \"e58ed647-be4f-4e08-b5e9-d919687aec38\") " pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:35:25.858329 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.858247    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmxn7\" (UniqueName: \"kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7\") pod \"network-check-target-wklxl\" (UID: \"0b9f399e-7383-4c4a-897c-fa71548f24d1\") " pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:35:25.862183 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.862157    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 11:35:25.862333 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.862212    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 11:35:25.871540 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.871516    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 11:35:25.871682 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.871662    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e58ed647-be4f-4e08-b5e9-d919687aec38-original-pull-secret\") pod \"global-pull-secret-syncer-nhrgs\" (UID: \"e58ed647-be4f-4e08-b5e9-d919687aec38\") " pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:35:25.881296 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.881272    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmxn7\" (UniqueName: \"kubernetes.io/projected/0b9f399e-7383-4c4a-897c-fa71548f24d1-kube-api-access-rmxn7\") pod \"network-check-target-wklxl\" (UID: \"0b9f399e-7383-4c4a-897c-fa71548f24d1\") " pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:35:25.900078 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.900027    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b45p4\""
Sep 11 11:35:25.906440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.906425    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:35:25.911973 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.911941    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nhrgs"
Sep 11 11:35:25.912062 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.912016    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tk98p\""
Sep 11 11:35:25.912130 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.912093    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:25.915069 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.915054    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2tbn7"
Sep 11 11:35:25.974086 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:25.974036    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:26.202192 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:26.202120    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wklxl"]
Sep 11 11:35:26.264663 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:35:26.264606    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b9f399e_7383_4c4a_897c_fa71548f24d1.slice/crio-737bf3dea58115e104fc161d7954728800ebaad98e2e0e3cd9d91f448ffb02b8 WatchSource:0}: Error finding container 737bf3dea58115e104fc161d7954728800ebaad98e2e0e3cd9d91f448ffb02b8: Status 404 returned error can't find the container with id 737bf3dea58115e104fc161d7954728800ebaad98e2e0e3cd9d91f448ffb02b8
Sep 11 11:35:26.383881 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:26.383852    2575 generic.go:356] "Generic (PLEG): container finished" podID="271013ff-ead3-4ebc-aa2e-cfc82bede0d7" containerID="9025e64d46cdec2cc439f36c6700c392f1d496826fb0bdcd19d4c5d357fb50fa" exitCode=0
Sep 11 11:35:26.384234 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:26.383921    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r8742" event={"ID":"271013ff-ead3-4ebc-aa2e-cfc82bede0d7","Type":"ContainerDied","Data":"9025e64d46cdec2cc439f36c6700c392f1d496826fb0bdcd19d4c5d357fb50fa"}
Sep 11 11:35:26.385198 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:26.385172    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wklxl" event={"ID":"0b9f399e-7383-4c4a-897c-fa71548f24d1","Type":"ContainerStarted","Data":"737bf3dea58115e104fc161d7954728800ebaad98e2e0e3cd9d91f448ffb02b8"}
Sep 11 11:35:26.571877 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:26.571857    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nhrgs"]
Sep 11 11:35:26.577917 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:26.577899    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2tbn7"]
Sep 11 11:35:26.585142 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:26.585124    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5978f55466-qxmnm"]
Sep 11 11:35:26.618095 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:26.618058    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:26.668975 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:35:26.668946    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode58ed647_be4f_4e08_b5e9_d919687aec38.slice/crio-717e4b24cae949bf2101d096116d7c6d336acdad8fd0b94ed2b72e962da980dc WatchSource:0}: Error finding container 717e4b24cae949bf2101d096116d7c6d336acdad8fd0b94ed2b72e962da980dc: Status 404 returned error can't find the container with id 717e4b24cae949bf2101d096116d7c6d336acdad8fd0b94ed2b72e962da980dc
Sep 11 11:35:26.669777 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:35:26.669755    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d81bb62_6b31_4915_8827_f8d7347665c0.slice/crio-9ed9ecb31c5bef3133d44c7d279d469dfdf57411142b968078fcb59515a2964e WatchSource:0}: Error finding container 9ed9ecb31c5bef3133d44c7d279d469dfdf57411142b968078fcb59515a2964e: Status 404 returned error can't find the container with id 9ed9ecb31c5bef3133d44c7d279d469dfdf57411142b968078fcb59515a2964e
Sep 11 11:35:26.670530 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:35:26.670506    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45118d7e_8d89_47ed_aa1a_47d58f96bb73.slice/crio-f9ad3c4b814a0f0c52049c1ab547b717aff92c1047f443ad7d1e67056b7abe7c WatchSource:0}: Error finding container f9ad3c4b814a0f0c52049c1ab547b717aff92c1047f443ad7d1e67056b7abe7c: Status 404 returned error can't find the container with id f9ad3c4b814a0f0c52049c1ab547b717aff92c1047f443ad7d1e67056b7abe7c
Sep 11 11:35:26.850365 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:26.850331    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:26.969674 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:26.969614    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:27.218493 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.218415    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7c76d5974f-5ntt7"]
Sep 11 11:35:27.235667 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.235250    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:27.237210 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.237142    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7c76d5974f-5ntt7"]
Sep 11 11:35:27.245482 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.245455    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 11 11:35:27.245482 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.245473    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-8359gb5n2tme4\""
Sep 11 11:35:27.246508 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.246488    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-pk4xv\""
Sep 11 11:35:27.246937 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.246916    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 11 11:35:27.247010 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.246984    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 11 11:35:27.247185 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.247167    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 11 11:35:27.331057 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.331004    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:27.337204 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.335675    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-dmr5s"]
Sep 11 11:35:27.338473 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.338446    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dmr5s"
Sep 11 11:35:27.342271 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.342236    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 11 11:35:27.342271 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.342241    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-tpnkw\""
Sep 11 11:35:27.373712 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.373679    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d62d9350-8ff9-4bb5-b38d-4145e6ecf12e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7c76d5974f-5ntt7\" (UID: \"d62d9350-8ff9-4bb5-b38d-4145e6ecf12e\") " pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:27.373858 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.373748    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d62d9350-8ff9-4bb5-b38d-4145e6ecf12e-secret-metrics-server-client-certs\") pod \"metrics-server-7c76d5974f-5ntt7\" (UID: \"d62d9350-8ff9-4bb5-b38d-4145e6ecf12e\") " pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:27.373858 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.373791    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d62d9350-8ff9-4bb5-b38d-4145e6ecf12e-client-ca-bundle\") pod \"metrics-server-7c76d5974f-5ntt7\" (UID: \"d62d9350-8ff9-4bb5-b38d-4145e6ecf12e\") " pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:27.373858 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.373816    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns5cl\" (UniqueName: \"kubernetes.io/projected/d62d9350-8ff9-4bb5-b38d-4145e6ecf12e-kube-api-access-ns5cl\") pod \"metrics-server-7c76d5974f-5ntt7\" (UID: \"d62d9350-8ff9-4bb5-b38d-4145e6ecf12e\") " pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:27.373858 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.373855    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d62d9350-8ff9-4bb5-b38d-4145e6ecf12e-secret-metrics-server-tls\") pod \"metrics-server-7c76d5974f-5ntt7\" (UID: \"d62d9350-8ff9-4bb5-b38d-4145e6ecf12e\") " pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:27.374079 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.373882    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d62d9350-8ff9-4bb5-b38d-4145e6ecf12e-audit-log\") pod \"metrics-server-7c76d5974f-5ntt7\" (UID: \"d62d9350-8ff9-4bb5-b38d-4145e6ecf12e\") " pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:27.374079 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.373927    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d62d9350-8ff9-4bb5-b38d-4145e6ecf12e-metrics-server-audit-profiles\") pod \"metrics-server-7c76d5974f-5ntt7\" (UID: \"d62d9350-8ff9-4bb5-b38d-4145e6ecf12e\") " pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:27.392286 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.392256    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm" event={"ID":"45118d7e-8d89-47ed-aa1a-47d58f96bb73","Type":"ContainerStarted","Data":"f9ad3c4b814a0f0c52049c1ab547b717aff92c1047f443ad7d1e67056b7abe7c"}
Sep 11 11:35:27.396235 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.396208    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-dmr5s"]
Sep 11 11:35:27.399152 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.399131    2575 generic.go:356] "Generic (PLEG): container finished" podID="bbe42078-01c1-49d6-a4df-785832fb726e" containerID="d114e504416e82c519031f0d9c3fa2722d9a1799701fd6d042e9f17b71f1e793" exitCode=0
Sep 11 11:35:27.400218 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.399219    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bbe42078-01c1-49d6-a4df-785832fb726e","Type":"ContainerDied","Data":"d114e504416e82c519031f0d9c3fa2722d9a1799701fd6d042e9f17b71f1e793"}
Sep 11 11:35:27.402030 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.401965    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2tbn7" event={"ID":"7d81bb62-6b31-4915-8827-f8d7347665c0","Type":"ContainerStarted","Data":"9ed9ecb31c5bef3133d44c7d279d469dfdf57411142b968078fcb59515a2964e"}
Sep 11 11:35:27.403870 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.403813    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nhrgs" event={"ID":"e58ed647-be4f-4e08-b5e9-d919687aec38","Type":"ContainerStarted","Data":"717e4b24cae949bf2101d096116d7c6d336acdad8fd0b94ed2b72e962da980dc"}
Sep 11 11:35:27.410714 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.410690    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r8742" event={"ID":"271013ff-ead3-4ebc-aa2e-cfc82bede0d7","Type":"ContainerStarted","Data":"fe663953344b9975222bda15e9ee587e854a45af5ec5169bd18f32c02222ea04"}
Sep 11 11:35:27.410823 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.410740    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r8742" event={"ID":"271013ff-ead3-4ebc-aa2e-cfc82bede0d7","Type":"ContainerStarted","Data":"a159c449362e84c584db83704e853c5e459f6f96a7cc449857f0ce73d80403d6"}
Sep 11 11:35:27.461676 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.461597    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:27.474427 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.474352    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d62d9350-8ff9-4bb5-b38d-4145e6ecf12e-secret-metrics-server-tls\") pod \"metrics-server-7c76d5974f-5ntt7\" (UID: \"d62d9350-8ff9-4bb5-b38d-4145e6ecf12e\") " pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:27.474427 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.474422    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d62d9350-8ff9-4bb5-b38d-4145e6ecf12e-audit-log\") pod \"metrics-server-7c76d5974f-5ntt7\" (UID: \"d62d9350-8ff9-4bb5-b38d-4145e6ecf12e\") " pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:27.474615 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.474480    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4rf8\" (UniqueName: \"kubernetes.io/projected/f77966fc-1af3-4d4c-92b7-3e1d50372792-kube-api-access-r4rf8\") pod \"monitoring-plugin-5d85778b9d-dmr5s\" (UID: \"f77966fc-1af3-4d4c-92b7-3e1d50372792\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dmr5s"
Sep 11 11:35:27.474615 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.474518    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d62d9350-8ff9-4bb5-b38d-4145e6ecf12e-metrics-server-audit-profiles\") pod \"metrics-server-7c76d5974f-5ntt7\" (UID: \"d62d9350-8ff9-4bb5-b38d-4145e6ecf12e\") " pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:27.474754 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.474665    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d62d9350-8ff9-4bb5-b38d-4145e6ecf12e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7c76d5974f-5ntt7\" (UID: \"d62d9350-8ff9-4bb5-b38d-4145e6ecf12e\") " pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:27.474814 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.474771    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d62d9350-8ff9-4bb5-b38d-4145e6ecf12e-secret-metrics-server-client-certs\") pod \"metrics-server-7c76d5974f-5ntt7\" (UID: \"d62d9350-8ff9-4bb5-b38d-4145e6ecf12e\") " pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:27.474814 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.474803    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f77966fc-1af3-4d4c-92b7-3e1d50372792-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-dmr5s\" (UID: \"f77966fc-1af3-4d4c-92b7-3e1d50372792\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dmr5s"
Sep 11 11:35:27.474910 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.474891    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d62d9350-8ff9-4bb5-b38d-4145e6ecf12e-client-ca-bundle\") pod \"metrics-server-7c76d5974f-5ntt7\" (UID: \"d62d9350-8ff9-4bb5-b38d-4145e6ecf12e\") " pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:27.474963 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.474919    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns5cl\" (UniqueName: \"kubernetes.io/projected/d62d9350-8ff9-4bb5-b38d-4145e6ecf12e-kube-api-access-ns5cl\") pod \"metrics-server-7c76d5974f-5ntt7\" (UID: \"d62d9350-8ff9-4bb5-b38d-4145e6ecf12e\") " pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:27.478058 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.476089    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d62d9350-8ff9-4bb5-b38d-4145e6ecf12e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7c76d5974f-5ntt7\" (UID: \"d62d9350-8ff9-4bb5-b38d-4145e6ecf12e\") " pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:27.478058 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.477778    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d62d9350-8ff9-4bb5-b38d-4145e6ecf12e-audit-log\") pod \"metrics-server-7c76d5974f-5ntt7\" (UID: \"d62d9350-8ff9-4bb5-b38d-4145e6ecf12e\") " pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:27.478058 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.477832    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d62d9350-8ff9-4bb5-b38d-4145e6ecf12e-metrics-server-audit-profiles\") pod \"metrics-server-7c76d5974f-5ntt7\" (UID: \"d62d9350-8ff9-4bb5-b38d-4145e6ecf12e\") " pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:27.479948 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.479907    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d62d9350-8ff9-4bb5-b38d-4145e6ecf12e-secret-metrics-server-client-certs\") pod \"metrics-server-7c76d5974f-5ntt7\" (UID: \"d62d9350-8ff9-4bb5-b38d-4145e6ecf12e\") " pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:27.483765 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.483719    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d62d9350-8ff9-4bb5-b38d-4145e6ecf12e-secret-metrics-server-tls\") pod \"metrics-server-7c76d5974f-5ntt7\" (UID: \"d62d9350-8ff9-4bb5-b38d-4145e6ecf12e\") " pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:27.489789 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.489749    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d62d9350-8ff9-4bb5-b38d-4145e6ecf12e-client-ca-bundle\") pod \"metrics-server-7c76d5974f-5ntt7\" (UID: \"d62d9350-8ff9-4bb5-b38d-4145e6ecf12e\") " pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:27.500484 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.500439    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns5cl\" (UniqueName: \"kubernetes.io/projected/d62d9350-8ff9-4bb5-b38d-4145e6ecf12e-kube-api-access-ns5cl\") pod \"metrics-server-7c76d5974f-5ntt7\" (UID: \"d62d9350-8ff9-4bb5-b38d-4145e6ecf12e\") " pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:27.506552 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.506472    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-r8742" podStartSLOduration=3.702540676 podStartE2EDuration="5.506454879s" totalImagesPullingTime="1.730384946s" totalInitContainerRuntime="73.529257ms" isStatefulPod=true podCreationTimestamp="2026-09-11 11:35:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:35:27.503777059 +0000 UTC m=+67.924383077" watchObservedRunningTime="2026-09-11 11:35:27.506454879 +0000 UTC m=+67.927060907"
Sep 11 11:35:27.556532 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.556485    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:27.575934 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.575903    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4rf8\" (UniqueName: \"kubernetes.io/projected/f77966fc-1af3-4d4c-92b7-3e1d50372792-kube-api-access-r4rf8\") pod \"monitoring-plugin-5d85778b9d-dmr5s\" (UID: \"f77966fc-1af3-4d4c-92b7-3e1d50372792\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dmr5s"
Sep 11 11:35:27.576079 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.575994    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f77966fc-1af3-4d4c-92b7-3e1d50372792-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-dmr5s\" (UID: \"f77966fc-1af3-4d4c-92b7-3e1d50372792\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dmr5s"
Sep 11 11:35:27.576356 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:27.576135    2575 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 11 11:35:27.576356 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:27.576202    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f77966fc-1af3-4d4c-92b7-3e1d50372792-monitoring-plugin-cert podName:f77966fc-1af3-4d4c-92b7-3e1d50372792 nodeName:}" failed. No retries permitted until 2026-09-11 11:35:28.076179962 +0000 UTC m=+68.496785982 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/f77966fc-1af3-4d4c-92b7-3e1d50372792-monitoring-plugin-cert") pod "monitoring-plugin-5d85778b9d-dmr5s" (UID: "f77966fc-1af3-4d4c-92b7-3e1d50372792") : secret "monitoring-plugin-cert" not found
Sep 11 11:35:27.603910 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.603848    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4rf8\" (UniqueName: \"kubernetes.io/projected/f77966fc-1af3-4d4c-92b7-3e1d50372792-kube-api-access-r4rf8\") pod \"monitoring-plugin-5d85778b9d-dmr5s\" (UID: \"f77966fc-1af3-4d4c-92b7-3e1d50372792\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dmr5s"
Sep 11 11:35:27.646235 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.646174    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65f55d9874-p6xzw"]
Sep 11 11:35:27.698812 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.698777    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68f64d9c64-w89kc"]
Sep 11 11:35:27.701595 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.701571    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:27.719944 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.719894    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68f64d9c64-w89kc"]
Sep 11 11:35:27.778939 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.778902    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/abb23749-d480-4ef5-94a8-ab11e3a86cc7-console-serving-cert\") pod \"console-68f64d9c64-w89kc\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") " pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:27.779131 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.778947    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvg4n\" (UniqueName: \"kubernetes.io/projected/abb23749-d480-4ef5-94a8-ab11e3a86cc7-kube-api-access-tvg4n\") pod \"console-68f64d9c64-w89kc\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") " pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:27.779131 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.779043    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-console-config\") pod \"console-68f64d9c64-w89kc\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") " pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:27.779131 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.779086    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-service-ca\") pod \"console-68f64d9c64-w89kc\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") " pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:27.779131 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.779117    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-oauth-serving-cert\") pod \"console-68f64d9c64-w89kc\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") " pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:27.779331 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.779175    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-trusted-ca-bundle\") pod \"console-68f64d9c64-w89kc\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") " pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:27.779331 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.779234    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/abb23749-d480-4ef5-94a8-ab11e3a86cc7-console-oauth-config\") pod \"console-68f64d9c64-w89kc\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") " pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:27.861707 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.861571    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7c76d5974f-5ntt7"]
Sep 11 11:35:27.881882 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.879952    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/abb23749-d480-4ef5-94a8-ab11e3a86cc7-console-oauth-config\") pod \"console-68f64d9c64-w89kc\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") " pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:27.881882 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.880036    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/abb23749-d480-4ef5-94a8-ab11e3a86cc7-console-serving-cert\") pod \"console-68f64d9c64-w89kc\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") " pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:27.881882 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.880054    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvg4n\" (UniqueName: \"kubernetes.io/projected/abb23749-d480-4ef5-94a8-ab11e3a86cc7-kube-api-access-tvg4n\") pod \"console-68f64d9c64-w89kc\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") " pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:27.881882 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.880090    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-console-config\") pod \"console-68f64d9c64-w89kc\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") " pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:27.881882 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.880117    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-service-ca\") pod \"console-68f64d9c64-w89kc\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") " pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:27.881882 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.880135    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-oauth-serving-cert\") pod \"console-68f64d9c64-w89kc\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") " pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:27.881882 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.880176    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-trusted-ca-bundle\") pod \"console-68f64d9c64-w89kc\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") " pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:27.881882 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.881132    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-trusted-ca-bundle\") pod \"console-68f64d9c64-w89kc\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") " pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:27.882455 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.882412    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-service-ca\") pod \"console-68f64d9c64-w89kc\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") " pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:27.883507 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.882999    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-console-config\") pod \"console-68f64d9c64-w89kc\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") " pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:27.883507 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.883471    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-oauth-serving-cert\") pod \"console-68f64d9c64-w89kc\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") " pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:27.885541 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.885498    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/abb23749-d480-4ef5-94a8-ab11e3a86cc7-console-serving-cert\") pod \"console-68f64d9c64-w89kc\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") " pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:27.888818 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.888722    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/abb23749-d480-4ef5-94a8-ab11e3a86cc7-console-oauth-config\") pod \"console-68f64d9c64-w89kc\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") " pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:27.902664 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.902614    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvg4n\" (UniqueName: \"kubernetes.io/projected/abb23749-d480-4ef5-94a8-ab11e3a86cc7-kube-api-access-tvg4n\") pod \"console-68f64d9c64-w89kc\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") " pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:27.923687 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.922505    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5c6968755b-x2kn6"]
Sep 11 11:35:27.927408 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.927314    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:27.934055 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.933077    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-tcbnk\""
Sep 11 11:35:27.934055 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.933273    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 11 11:35:27.934055 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.933494    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 11 11:35:27.934055 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.933712    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 11 11:35:27.934390 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.934077    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 11 11:35:27.934469 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.934395    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:27.935850 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.934669    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 11 11:35:27.943459 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.943241    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 11 11:35:27.957890 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:27.957810    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5c6968755b-x2kn6"]
Sep 11 11:35:28.014123 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.014092    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:28.078301 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:35:28.078220    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd62d9350_8ff9_4bb5_b38d_4145e6ecf12e.slice/crio-b820acd484d28aaa9a3752972a1bf9df2aac0111f35373d29c74f6c2cda9bae8 WatchSource:0}: Error finding container b820acd484d28aaa9a3752972a1bf9df2aac0111f35373d29c74f6c2cda9bae8: Status 404 returned error can't find the container with id b820acd484d28aaa9a3752972a1bf9df2aac0111f35373d29c74f6c2cda9bae8
Sep 11 11:35:28.083813 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.083235    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/addb1856-5092-45ff-81c2-9a0b0082eb42-telemeter-client-tls\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.083813 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.083284    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/addb1856-5092-45ff-81c2-9a0b0082eb42-metrics-client-ca\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.083813 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.083393    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pth7\" (UniqueName: \"kubernetes.io/projected/addb1856-5092-45ff-81c2-9a0b0082eb42-kube-api-access-4pth7\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.083813 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.083490    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/addb1856-5092-45ff-81c2-9a0b0082eb42-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.083813 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.083572    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/addb1856-5092-45ff-81c2-9a0b0082eb42-secret-telemeter-client\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.083813 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.083606    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/addb1856-5092-45ff-81c2-9a0b0082eb42-serving-certs-ca-bundle\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.086350 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.085781    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/addb1856-5092-45ff-81c2-9a0b0082eb42-federate-client-tls\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.086350 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.085828    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/addb1856-5092-45ff-81c2-9a0b0082eb42-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.086350 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.085902    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f77966fc-1af3-4d4c-92b7-3e1d50372792-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-dmr5s\" (UID: \"f77966fc-1af3-4d4c-92b7-3e1d50372792\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dmr5s"
Sep 11 11:35:28.095237 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.095184    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f77966fc-1af3-4d4c-92b7-3e1d50372792-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-dmr5s\" (UID: \"f77966fc-1af3-4d4c-92b7-3e1d50372792\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dmr5s"
Sep 11 11:35:28.188587 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.187394    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/addb1856-5092-45ff-81c2-9a0b0082eb42-serving-certs-ca-bundle\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.188587 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.187448    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/addb1856-5092-45ff-81c2-9a0b0082eb42-federate-client-tls\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.188587 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.187477    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/addb1856-5092-45ff-81c2-9a0b0082eb42-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.188587 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.187558    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/addb1856-5092-45ff-81c2-9a0b0082eb42-telemeter-client-tls\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.188587 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.187592    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/addb1856-5092-45ff-81c2-9a0b0082eb42-metrics-client-ca\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.188587 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.187716    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pth7\" (UniqueName: \"kubernetes.io/projected/addb1856-5092-45ff-81c2-9a0b0082eb42-kube-api-access-4pth7\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.188587 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.187770    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/addb1856-5092-45ff-81c2-9a0b0082eb42-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.188587 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.187821    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/addb1856-5092-45ff-81c2-9a0b0082eb42-secret-telemeter-client\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.191593 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.189908    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/addb1856-5092-45ff-81c2-9a0b0082eb42-metrics-client-ca\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.191593 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.191272    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/addb1856-5092-45ff-81c2-9a0b0082eb42-serving-certs-ca-bundle\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.192212 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.192190    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/addb1856-5092-45ff-81c2-9a0b0082eb42-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.196422 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.196381    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/addb1856-5092-45ff-81c2-9a0b0082eb42-telemeter-client-tls\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.197169 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.197126    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/addb1856-5092-45ff-81c2-9a0b0082eb42-secret-telemeter-client\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.197401 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.197382    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/addb1856-5092-45ff-81c2-9a0b0082eb42-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.203479 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.203457    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/addb1856-5092-45ff-81c2-9a0b0082eb42-federate-client-tls\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.217754 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.217727    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pth7\" (UniqueName: \"kubernetes.io/projected/addb1856-5092-45ff-81c2-9a0b0082eb42-kube-api-access-4pth7\") pod \"telemeter-client-5c6968755b-x2kn6\" (UID: \"addb1856-5092-45ff-81c2-9a0b0082eb42\") " pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.249999 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.249758    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6"
Sep 11 11:35:28.252199 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.251995    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dmr5s"
Sep 11 11:35:28.384111 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.383971    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68f64d9c64-w89kc"]
Sep 11 11:35:28.394718 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:35:28.394660    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabb23749_d480_4ef5_94a8_ab11e3a86cc7.slice/crio-2e026a8a7024ca66fc73696341fa618c5262719a402f70178e7beba10799693f WatchSource:0}: Error finding container 2e026a8a7024ca66fc73696341fa618c5262719a402f70178e7beba10799693f: Status 404 returned error can't find the container with id 2e026a8a7024ca66fc73696341fa618c5262719a402f70178e7beba10799693f
Sep 11 11:35:28.430100 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.430029    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68f64d9c64-w89kc" event={"ID":"abb23749-d480-4ef5-94a8-ab11e3a86cc7","Type":"ContainerStarted","Data":"2e026a8a7024ca66fc73696341fa618c5262719a402f70178e7beba10799693f"}
Sep 11 11:35:28.434077 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.434011    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7" event={"ID":"d62d9350-8ff9-4bb5-b38d-4145e6ecf12e","Type":"ContainerStarted","Data":"b820acd484d28aaa9a3752972a1bf9df2aac0111f35373d29c74f6c2cda9bae8"}
Sep 11 11:35:28.437212 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.437148    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2tbn7" event={"ID":"7d81bb62-6b31-4915-8827-f8d7347665c0","Type":"ContainerStarted","Data":"c00f699918ae0f9ecba0840d7d8a84ab9fd83648e3c7cff22c4a68e38b10487c"}
Sep 11 11:35:28.576438 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.576343    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5c6968755b-x2kn6"]
Sep 11 11:35:28.581802 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:35:28.581734    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaddb1856_5092_45ff_81c2_9a0b0082eb42.slice/crio-7622349af49f4a0cfe98fbd075976d48b767c8af9fccd61c0d97e39e6134376b WatchSource:0}: Error finding container 7622349af49f4a0cfe98fbd075976d48b767c8af9fccd61c0d97e39e6134376b: Status 404 returned error can't find the container with id 7622349af49f4a0cfe98fbd075976d48b767c8af9fccd61c0d97e39e6134376b
Sep 11 11:35:28.607424 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.607353    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-dmr5s"]
Sep 11 11:35:28.891905 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.883557    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:35:28.891905 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.884902    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:28.891905 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.887453    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:28.898997 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.898750    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 11:35:28.898997 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.898809    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 11:35:28.898997 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.898863    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 11:35:28.903423 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.902206    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 11:35:28.903423 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.902538    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 11:35:28.903423 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.903376    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 11:35:28.903930 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.903617    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 11:35:28.912302 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.905484    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 11:35:28.912302 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.905766    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fv1qua16vuedl\""
Sep 11 11:35:28.915448 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.915025    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 11:35:28.916577 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.916539    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-jx9nk\""
Sep 11 11:35:28.917604 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.917056    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 11:35:28.925239 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.925208    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 11:35:28.925392 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.925373    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 11:35:28.926907 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.926884    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 11:35:28.927110 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.927029    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:35:28.931925 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.931899    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 11:35:28.996818 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.996769    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:28.996818 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.996820    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44d30cae-3e69-4694-b8ab-5dd645d4e676-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:28.997056 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.996849    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:28.997056 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.997020    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:28.997056 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.997051    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:28.997202 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.997074    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:28.997202 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.997104    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:28.997202 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.997135    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:28.997345 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.997244    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:28.997345 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.997303    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:28.997345 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.997335    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-web-config\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:28.997489 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.997357    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rvrw\" (UniqueName: \"kubernetes.io/projected/44d30cae-3e69-4694-b8ab-5dd645d4e676-kube-api-access-8rvrw\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:28.997489 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.997441    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:28.997489 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.997467    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:28.997655 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.997497    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:28.997655 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.997529    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44d30cae-3e69-4694-b8ab-5dd645d4e676-config-out\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:28.997655 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.997568    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:28.997655 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.997593    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:28.997655 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.997618    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-config\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:28.997904 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:28.997663    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.099111 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.099071    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.099292 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.099122    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.099292 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.099148    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.099292 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.099180    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.099292 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.099202    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.099292 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.099228    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.100838 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.099979    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.100838 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.100197    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.100838 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.100431    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.100838 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.100464    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-web-config\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.100838 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.100489    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rvrw\" (UniqueName: \"kubernetes.io/projected/44d30cae-3e69-4694-b8ab-5dd645d4e676-kube-api-access-8rvrw\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.100838 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.100555    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.100838 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.100583    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.100838 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.100619    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.100838 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.100671    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44d30cae-3e69-4694-b8ab-5dd645d4e676-config-out\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.100838 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.100696    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.100838 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.100718    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.100838 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.100743    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-config\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.100838 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.100769    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.100838 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.100796    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.100838 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.100831    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44d30cae-3e69-4694-b8ab-5dd645d4e676-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.101590 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.100857    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.101590 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.100897    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.101590 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.101174    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.101826 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.101802    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.102376 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.102156    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.102376 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.102359    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.105872 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.105732    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.106699 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.106337    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.108488 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.108466    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.109874 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.109852    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.112655 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.112205    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.112655 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.112589    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.113778 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.113704    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.114055 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.113927    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44d30cae-3e69-4694-b8ab-5dd645d4e676-config-out\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.114554 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.114532    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.116238 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.114946    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-web-config\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.116238 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.115574    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rvrw\" (UniqueName: \"kubernetes.io/projected/44d30cae-3e69-4694-b8ab-5dd645d4e676-kube-api-access-8rvrw\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.119348 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.119270    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-config\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.119798 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.119753    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44d30cae-3e69-4694-b8ab-5dd645d4e676-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.230264 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.229850    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:29.446351 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.446297    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2tbn7" event={"ID":"7d81bb62-6b31-4915-8827-f8d7347665c0","Type":"ContainerStarted","Data":"9c47f07298dacc080e12bbb99c54cbe87d105b891bf1c8fa49e43a9351f4f93b"}
Sep 11 11:35:29.453564 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.453530    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6" event={"ID":"addb1856-5092-45ff-81c2-9a0b0082eb42","Type":"ContainerStarted","Data":"7622349af49f4a0cfe98fbd075976d48b767c8af9fccd61c0d97e39e6134376b"}
Sep 11 11:35:29.456334 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.456303    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68f64d9c64-w89kc" event={"ID":"abb23749-d480-4ef5-94a8-ab11e3a86cc7","Type":"ContainerStarted","Data":"12d4199ae80f40cddc16cf5bcbdae3f97c2db4a719187d46ee92fb144b31179e"}
Sep 11 11:35:29.459003 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.458975    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dmr5s" event={"ID":"f77966fc-1af3-4d4c-92b7-3e1d50372792","Type":"ContainerStarted","Data":"a24a6c934ea969c0178286b338ce71c945fbfca227f0cc66b804660b9fcd59db"}
Sep 11 11:35:29.468730 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.468678    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2tbn7" podStartSLOduration=68.059151544 podStartE2EDuration="1m9.46865866s" totalImagesPullingTime="1.409507116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:34:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:35:29.468196085 +0000 UTC m=+69.888802113" watchObservedRunningTime="2026-09-11 11:35:29.46865866 +0000 UTC m=+69.889264684"
Sep 11 11:35:29.490560 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.490508    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68f64d9c64-w89kc" podStartSLOduration=2.49048491 podStartE2EDuration="2.49048491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:35:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:35:29.489291016 +0000 UTC m=+69.909897044" watchObservedRunningTime="2026-09-11 11:35:29.49048491 +0000 UTC m=+69.911090935"
Sep 11 11:35:29.580295 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.580253    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:29.678964 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:29.678922    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:30.351388 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:30.351359    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-k6szw"
Sep 11 11:35:30.560588 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:30.560510    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:30.669309 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:30.669207    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:30.786545 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:30.786508    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:30.968923 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:30.968835    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:31.070822 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:31.070778    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:31.902315 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:31.902284    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:32.007482 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:32.007452    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:32.250227 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:32.250192    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:33.545437 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:33.545411    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:35:34.018950 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:34.018912    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:34.170468 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:35:34.170437    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44d30cae_3e69_4694_b8ab_5dd645d4e676.slice/crio-f5a3794d2d4cdbffccdfd6773b59b0e40bf1e3975993b9b6ce64fe2d119dda55 WatchSource:0}: Error finding container f5a3794d2d4cdbffccdfd6773b59b0e40bf1e3975993b9b6ce64fe2d119dda55: Status 404 returned error can't find the container with id f5a3794d2d4cdbffccdfd6773b59b0e40bf1e3975993b9b6ce64fe2d119dda55
Sep 11 11:35:34.197468 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:34.197440    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:34.487212 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:34.487182    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bbe42078-01c1-49d6-a4df-785832fb726e","Type":"ContainerStarted","Data":"f502523aae764264ec2c062e09223b2a9d68fa3f7e271ae78b47606a49eed818"}
Sep 11 11:35:34.488908 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:34.488880    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7" event={"ID":"d62d9350-8ff9-4bb5-b38d-4145e6ecf12e","Type":"ContainerStarted","Data":"421f2d03b9ce908e6c9c3e177291c45ecc0deddd22f1ecb917b62009799a59b1"}
Sep 11 11:35:34.490667 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:34.490458    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dmr5s" event={"ID":"f77966fc-1af3-4d4c-92b7-3e1d50372792","Type":"ContainerStarted","Data":"1559083dccf7d67bed732504c89aa9be4e912d1a088d1ac535c5f42479adea01"}
Sep 11 11:35:34.490859 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:34.490800    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dmr5s"
Sep 11 11:35:34.492923 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:34.492903    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nhrgs" event={"ID":"e58ed647-be4f-4e08-b5e9-d919687aec38","Type":"ContainerStarted","Data":"144cb65225feddab0ce4c5e4830f281f608b8d8fa06c298bcfe2a219eb47f956"}
Sep 11 11:35:34.496855 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:34.495715    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm" event={"ID":"45118d7e-8d89-47ed-aa1a-47d58f96bb73","Type":"ContainerStarted","Data":"4c025fe8bc19c1749173e6a38d97c5991f6adb7bac589832bd17a2932b3a4be4"}
Sep 11 11:35:34.498478 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:34.498322    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dmr5s"
Sep 11 11:35:34.500589 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:34.500101    2575 generic.go:356] "Generic (PLEG): container finished" podID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerID="35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4" exitCode=0
Sep 11 11:35:34.500589 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:34.500221    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44d30cae-3e69-4694-b8ab-5dd645d4e676","Type":"ContainerDied","Data":"35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4"}
Sep 11 11:35:34.500589 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:34.500300    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44d30cae-3e69-4694-b8ab-5dd645d4e676","Type":"ContainerStarted","Data":"f5a3794d2d4cdbffccdfd6773b59b0e40bf1e3975993b9b6ce64fe2d119dda55"}
Sep 11 11:35:34.505837 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:34.504427    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6" event={"ID":"addb1856-5092-45ff-81c2-9a0b0082eb42","Type":"ContainerStarted","Data":"c5990f21090ab276bfb6b872e3a67475c3c09d2752ba4afd593603d91177cf7c"}
Sep 11 11:35:34.507730 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:34.507483    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wklxl" event={"ID":"0b9f399e-7383-4c4a-897c-fa71548f24d1","Type":"ContainerStarted","Data":"19b51d6f4d526f59633a8e21cac386e2755de17ab3bb0125a21c8c42b2a0c9d5"}
Sep 11 11:35:34.507867 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:34.507818    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:35:34.526008 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:34.525085    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7" podStartSLOduration=1.381830018 podStartE2EDuration="7.525065762s" totalImagesPullingTime="6.143235744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:35:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:35:34.525054425 +0000 UTC m=+74.945660454" watchObservedRunningTime="2026-09-11 11:35:34.525065762 +0000 UTC m=+74.945671789"
Sep 11 11:35:34.557531 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:34.556925    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wklxl" podStartSLOduration=67.597633289 podStartE2EDuration="1m15.556903154s" totalImagesPullingTime="7.959269865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:34:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:35:34.55641226 +0000 UTC m=+74.977018289" watchObservedRunningTime="2026-09-11 11:35:34.556903154 +0000 UTC m=+74.977509179"
Sep 11 11:35:34.593015 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:34.592959    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nhrgs" podStartSLOduration=67.064262024 podStartE2EDuration="1m14.592936206s" totalImagesPullingTime="7.528674182s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 11:34:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:35:34.592220822 +0000 UTC m=+75.012826851" watchObservedRunningTime="2026-09-11 11:35:34.592936206 +0000 UTC m=+75.013542234"
Sep 11 11:35:34.636294 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:34.636015    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-dmr5s" podStartSLOduration=2.021100814 podStartE2EDuration="7.635993364s" totalImagesPullingTime="5.61489255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:35:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:35:34.63502562 +0000 UTC m=+75.055631649" watchObservedRunningTime="2026-09-11 11:35:34.635993364 +0000 UTC m=+75.056599393"
Sep 11 11:35:35.517566 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:35.517529    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bbe42078-01c1-49d6-a4df-785832fb726e","Type":"ContainerStarted","Data":"028afa777b7d5e590ec4e9d2a6679183aa7acdaefaa2e786c6dae28492f77176"}
Sep 11 11:35:35.517698 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:35.517619    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bbe42078-01c1-49d6-a4df-785832fb726e","Type":"ContainerStarted","Data":"41a1d65662827f974b103a9438ed149dac3f326847a814b259607a2962310f9a"}
Sep 11 11:35:35.517759 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:35.517709    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bbe42078-01c1-49d6-a4df-785832fb726e","Type":"ContainerStarted","Data":"0b29a4308bfe45d5be18034a89c220d246d44192830fd02183e95072379d66db"}
Sep 11 11:35:35.517759 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:35.517749    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bbe42078-01c1-49d6-a4df-785832fb726e","Type":"ContainerStarted","Data":"f4b182b56923c0468e8dd68094e80d747e0a40192f51f28e153a016dc8720b3a"}
Sep 11 11:35:35.520855 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:35.520735    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm" event={"ID":"45118d7e-8d89-47ed-aa1a-47d58f96bb73","Type":"ContainerStarted","Data":"f3b068a072a11c2b0a8456e2c1149be4bd810ee955ec79dd4df1255f8a323b4b"}
Sep 11 11:35:35.520855 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:35.520805    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm" event={"ID":"45118d7e-8d89-47ed-aa1a-47d58f96bb73","Type":"ContainerStarted","Data":"f0ed31672582a65cebcfb881e266cfcb2411b212b9166610d10100d10882b5b5"}
Sep 11 11:35:35.529571 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:35.529539    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6" event={"ID":"addb1856-5092-45ff-81c2-9a0b0082eb42","Type":"ContainerStarted","Data":"3759c6a2cea541449a77ebd405133023ea1b6979f541e03e9f6d7865d910b4f0"}
Sep 11 11:35:35.529775 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:35.529695    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6" event={"ID":"addb1856-5092-45ff-81c2-9a0b0082eb42","Type":"ContainerStarted","Data":"830bd8ec52cbf46631af4e8bbcb8e79f9d4fa441e0979d1a072b13612654438c"}
Sep 11 11:35:35.567825 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:35.567774    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5c6968755b-x2kn6" podStartSLOduration=2.91763598 podStartE2EDuration="8.567753276s" totalImagesPullingTime="5.650117296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:35:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:35:35.566786164 +0000 UTC m=+75.987392194" watchObservedRunningTime="2026-09-11 11:35:35.567753276 +0000 UTC m=+75.988359304"
Sep 11 11:35:35.751001 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:35.750973    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68f64d9c64-w89kc"]
Sep 11 11:35:35.840880 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:35.840842    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7df75766fd-lfdzx"]
Sep 11 11:35:35.844559 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:35.844537    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:35.867594 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:35.867533    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7df75766fd-lfdzx"]
Sep 11 11:35:35.972908 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:35.972877    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6457a6e8-f976-416c-a169-4f51b07be349-console-oauth-config\") pod \"console-7df75766fd-lfdzx\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") " pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:35.972908 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:35.972907    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hshxf\" (UniqueName: \"kubernetes.io/projected/6457a6e8-f976-416c-a169-4f51b07be349-kube-api-access-hshxf\") pod \"console-7df75766fd-lfdzx\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") " pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:35.973113 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:35.972970    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-oauth-serving-cert\") pod \"console-7df75766fd-lfdzx\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") " pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:35.973113 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:35.972998    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-service-ca\") pod \"console-7df75766fd-lfdzx\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") " pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:35.973113 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:35.973036    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6457a6e8-f976-416c-a169-4f51b07be349-console-serving-cert\") pod \"console-7df75766fd-lfdzx\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") " pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:35.973113 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:35.973093    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-trusted-ca-bundle\") pod \"console-7df75766fd-lfdzx\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") " pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:35.973266 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:35.973138    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-console-config\") pod \"console-7df75766fd-lfdzx\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") " pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:36.074101 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.074006    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6457a6e8-f976-416c-a169-4f51b07be349-console-oauth-config\") pod \"console-7df75766fd-lfdzx\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") " pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:36.074101 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.074053    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hshxf\" (UniqueName: \"kubernetes.io/projected/6457a6e8-f976-416c-a169-4f51b07be349-kube-api-access-hshxf\") pod \"console-7df75766fd-lfdzx\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") " pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:36.074101 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.074091    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-oauth-serving-cert\") pod \"console-7df75766fd-lfdzx\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") " pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:36.074318 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.074119    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-service-ca\") pod \"console-7df75766fd-lfdzx\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") " pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:36.074522 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.074480    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6457a6e8-f976-416c-a169-4f51b07be349-console-serving-cert\") pod \"console-7df75766fd-lfdzx\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") " pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:36.074680 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.074539    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-trusted-ca-bundle\") pod \"console-7df75766fd-lfdzx\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") " pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:36.074680 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.074576    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-console-config\") pod \"console-7df75766fd-lfdzx\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") " pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:36.075024 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.074971    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-oauth-serving-cert\") pod \"console-7df75766fd-lfdzx\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") " pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:36.075334 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.075310    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-console-config\") pod \"console-7df75766fd-lfdzx\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") " pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:36.075420 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.075337    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-service-ca\") pod \"console-7df75766fd-lfdzx\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") " pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:36.075721 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.075701    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-trusted-ca-bundle\") pod \"console-7df75766fd-lfdzx\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") " pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:36.076593 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.076576    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6457a6e8-f976-416c-a169-4f51b07be349-console-oauth-config\") pod \"console-7df75766fd-lfdzx\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") " pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:36.080354 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.080311    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6457a6e8-f976-416c-a169-4f51b07be349-console-serving-cert\") pod \"console-7df75766fd-lfdzx\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") " pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:36.100909 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.100882    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hshxf\" (UniqueName: \"kubernetes.io/projected/6457a6e8-f976-416c-a169-4f51b07be349-kube-api-access-hshxf\") pod \"console-7df75766fd-lfdzx\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") " pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:36.155405 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.155377    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:36.359605 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.359466    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:36.481041 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.480997    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7df75766fd-lfdzx"]
Sep 11 11:35:36.484816 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:35:36.484794    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6457a6e8_f976_416c_a169_4f51b07be349.slice/crio-40e4998e892f59532545a640acab1a992903029ece1b4df6c6ca022363f60889 WatchSource:0}: Error finding container 40e4998e892f59532545a640acab1a992903029ece1b4df6c6ca022363f60889: Status 404 returned error can't find the container with id 40e4998e892f59532545a640acab1a992903029ece1b4df6c6ca022363f60889
Sep 11 11:35:36.534078 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.534045    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7df75766fd-lfdzx" event={"ID":"6457a6e8-f976-416c-a169-4f51b07be349","Type":"ContainerStarted","Data":"40e4998e892f59532545a640acab1a992903029ece1b4df6c6ca022363f60889"}
Sep 11 11:35:36.545363 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.545335    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:36.547671 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.547379    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm" event={"ID":"45118d7e-8d89-47ed-aa1a-47d58f96bb73","Type":"ContainerStarted","Data":"2180ff31814e5bb33747a4f1dbaaa54c2472fe2c880f65a344205c018c3534a6"}
Sep 11 11:35:36.547671 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.547468    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm" event={"ID":"45118d7e-8d89-47ed-aa1a-47d58f96bb73","Type":"ContainerStarted","Data":"737bfb9253865c738f8c96857af1804142c3ec14bdef010658b00a1ecf30a9bd"}
Sep 11 11:35:36.547925 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.547905    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:36.547989 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.547969    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm" event={"ID":"45118d7e-8d89-47ed-aa1a-47d58f96bb73","Type":"ContainerStarted","Data":"fa97c6151aeda3f05319d4b03a1ec20400d1bf59caad45be2fdbc8e9017c8a4c"}
Sep 11 11:35:36.614607 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:36.614485    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm" podStartSLOduration=4.188081079 podStartE2EDuration="11.614468406s" totalImagesPullingTime="7.426387327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:35:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:35:36.611932758 +0000 UTC m=+77.032538788" watchObservedRunningTime="2026-09-11 11:35:36.614468406 +0000 UTC m=+77.035074432"
Sep 11 11:35:37.554452 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:37.554417    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7df75766fd-lfdzx" event={"ID":"6457a6e8-f976-416c-a169-4f51b07be349","Type":"ContainerStarted","Data":"3e52c3436968468bc97cb5bd1de90c3b3dca70bc5d27f27133b30940d0bf745c"}
Sep 11 11:35:37.558510 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:37.558477    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bbe42078-01c1-49d6-a4df-785832fb726e","Type":"ContainerStarted","Data":"adfaf34b6db0b7c7fa79ffabafca31d3828ee038f7462d45d4bf658d9941e723"}
Sep 11 11:35:37.585781 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:37.585727    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7df75766fd-lfdzx" podStartSLOduration=2.585707327 podStartE2EDuration="2.585707327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:35:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:35:37.584801454 +0000 UTC m=+78.005407481" watchObservedRunningTime="2026-09-11 11:35:37.585707327 +0000 UTC m=+78.006313354"
Sep 11 11:35:38.014726 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:38.014705    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:35:38.567395 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:38.567352    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44d30cae-3e69-4694-b8ab-5dd645d4e676","Type":"ContainerStarted","Data":"56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731"}
Sep 11 11:35:38.567547 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:38.567424    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44d30cae-3e69-4694-b8ab-5dd645d4e676","Type":"ContainerStarted","Data":"ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0"}
Sep 11 11:35:38.567547 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:38.567465    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44d30cae-3e69-4694-b8ab-5dd645d4e676","Type":"ContainerStarted","Data":"0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d"}
Sep 11 11:35:38.567547 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:38.567490    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44d30cae-3e69-4694-b8ab-5dd645d4e676","Type":"ContainerStarted","Data":"e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910"}
Sep 11 11:35:38.567547 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:38.567513    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44d30cae-3e69-4694-b8ab-5dd645d4e676","Type":"ContainerStarted","Data":"a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872"}
Sep 11 11:35:38.567547 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:38.567546    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44d30cae-3e69-4694-b8ab-5dd645d4e676","Type":"ContainerStarted","Data":"172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf"}
Sep 11 11:35:38.607374 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:38.607312    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=5.407306305 podStartE2EDuration="15.607286345s" totalImagesPullingTime="10.186956651s" totalInitContainerRuntime="13.023389ms" isStatefulPod=false podCreationTimestamp="2026-09-11 11:35:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:35:37.653796913 +0000 UTC m=+78.074402940" watchObservedRunningTime="2026-09-11 11:35:38.607286345 +0000 UTC m=+79.027892374"
Sep 11 11:35:38.609431 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:38.609381    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=7.228554813 podStartE2EDuration="10.609362475s" totalImagesPullingTime="3.352236494s" totalInitContainerRuntime="28.571168ms" isStatefulPod=false podCreationTimestamp="2026-09-11 11:35:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:35:38.599990773 +0000 UTC m=+79.020596799" watchObservedRunningTime="2026-09-11 11:35:38.609362475 +0000 UTC m=+79.029968502"
Sep 11 11:35:39.230751 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:39.230708    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:35:40.370991 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:40.370955    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:40.501378 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:40.501347    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:41.350777 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:41.350745    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:41.452446 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:41.452330    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:42.565366 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:42.565341    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5978f55466-qxmnm"
Sep 11 11:35:46.156530 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:46.156497    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:46.156967 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:46.156800    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:46.161337 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:46.161316    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:46.606676 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:46.606649    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:35:46.700312 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:46.700282    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c7c544647-2qvfm"]
Sep 11 11:35:46.917391 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:46.917309    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:47.021754 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:47.021720    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:47.557092 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:47.557056    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:47.557465 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:47.557116    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:35:48.980100 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:48.980060    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:49.081218 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:49.081182    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:52.678543 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:52.678506    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-65f55d9874-p6xzw" podUID="5d08cc13-d976-4959-8fc7-a851d6d4c82d" containerName="console" containerID="cri-o://81bf5aeab9607d8ce5dea533dc0dfb4b471c1541e963e6cd5de9b2869135441d" gracePeriod=15
Sep 11 11:35:52.926834 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:52.926812    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:53.030206 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.030173    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-console-config\") pod \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") "
Sep 11 11:35:53.030384 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.030240    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bdhk\" (UniqueName: \"kubernetes.io/projected/5d08cc13-d976-4959-8fc7-a851d6d4c82d-kube-api-access-5bdhk\") pod \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") "
Sep 11 11:35:53.030384 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.030299    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d08cc13-d976-4959-8fc7-a851d6d4c82d-console-oauth-config\") pod \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") "
Sep 11 11:35:53.030384 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.030328    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-oauth-serving-cert\") pod \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") "
Sep 11 11:35:53.030384 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.030367    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d08cc13-d976-4959-8fc7-a851d6d4c82d-console-serving-cert\") pod \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") "
Sep 11 11:35:53.030592 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.030404    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-trusted-ca-bundle\") pod \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") "
Sep 11 11:35:53.030592 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.030459    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-service-ca\") pod \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\" (UID: \"5d08cc13-d976-4959-8fc7-a851d6d4c82d\") "
Sep 11 11:35:53.030781 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.030750    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-console-config" (OuterVolumeSpecName: "console-config") pod "5d08cc13-d976-4959-8fc7-a851d6d4c82d" (UID: "5d08cc13-d976-4959-8fc7-a851d6d4c82d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:35:53.030997 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.030780    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5d08cc13-d976-4959-8fc7-a851d6d4c82d" (UID: "5d08cc13-d976-4959-8fc7-a851d6d4c82d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:35:53.031077 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.031010    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5d08cc13-d976-4959-8fc7-a851d6d4c82d" (UID: "5d08cc13-d976-4959-8fc7-a851d6d4c82d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:35:53.031077 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.031037    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-service-ca" (OuterVolumeSpecName: "service-ca") pod "5d08cc13-d976-4959-8fc7-a851d6d4c82d" (UID: "5d08cc13-d976-4959-8fc7-a851d6d4c82d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:35:53.032732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.032707    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d08cc13-d976-4959-8fc7-a851d6d4c82d-kube-api-access-5bdhk" (OuterVolumeSpecName: "kube-api-access-5bdhk") pod "5d08cc13-d976-4959-8fc7-a851d6d4c82d" (UID: "5d08cc13-d976-4959-8fc7-a851d6d4c82d"). InnerVolumeSpecName "kube-api-access-5bdhk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:35:53.033225 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.033201    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d08cc13-d976-4959-8fc7-a851d6d4c82d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5d08cc13-d976-4959-8fc7-a851d6d4c82d" (UID: "5d08cc13-d976-4959-8fc7-a851d6d4c82d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:35:53.033282 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.033226    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d08cc13-d976-4959-8fc7-a851d6d4c82d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5d08cc13-d976-4959-8fc7-a851d6d4c82d" (UID: "5d08cc13-d976-4959-8fc7-a851d6d4c82d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:35:53.131751 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.131712    2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-trusted-ca-bundle\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:35:53.131751 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.131748    2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-service-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:35:53.131751 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.131756    2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-console-config\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:35:53.131967 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.131764    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bdhk\" (UniqueName: \"kubernetes.io/projected/5d08cc13-d976-4959-8fc7-a851d6d4c82d-kube-api-access-5bdhk\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:35:53.131967 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.131774    2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5d08cc13-d976-4959-8fc7-a851d6d4c82d-console-oauth-config\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:35:53.131967 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.131782    2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5d08cc13-d976-4959-8fc7-a851d6d4c82d-oauth-serving-cert\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:35:53.131967 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.131790    2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5d08cc13-d976-4959-8fc7-a851d6d4c82d-console-serving-cert\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:35:53.634128 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.634093    2575 generic.go:356] "Generic (PLEG): container finished" podID="5d08cc13-d976-4959-8fc7-a851d6d4c82d" containerID="81bf5aeab9607d8ce5dea533dc0dfb4b471c1541e963e6cd5de9b2869135441d" exitCode=0
Sep 11 11:35:53.634286 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.634138    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65f55d9874-p6xzw" event={"ID":"5d08cc13-d976-4959-8fc7-a851d6d4c82d","Type":"ContainerDied","Data":"81bf5aeab9607d8ce5dea533dc0dfb4b471c1541e963e6cd5de9b2869135441d"}
Sep 11 11:35:53.634286 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.634151    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65f55d9874-p6xzw"
Sep 11 11:35:53.634286 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.634177    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65f55d9874-p6xzw" event={"ID":"5d08cc13-d976-4959-8fc7-a851d6d4c82d","Type":"ContainerDied","Data":"efd3ea5d4adffd359d8eb93b0def9c6f1b335c792903c36a5ecb67a98b90e1b4"}
Sep 11 11:35:53.634286 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.634196    2575 scope.go:117] "RemoveContainer" containerID="81bf5aeab9607d8ce5dea533dc0dfb4b471c1541e963e6cd5de9b2869135441d"
Sep 11 11:35:53.643842 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.643822    2575 scope.go:117] "RemoveContainer" containerID="81bf5aeab9607d8ce5dea533dc0dfb4b471c1541e963e6cd5de9b2869135441d"
Sep 11 11:35:53.644098 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:35:53.644080    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81bf5aeab9607d8ce5dea533dc0dfb4b471c1541e963e6cd5de9b2869135441d\": container with ID starting with 81bf5aeab9607d8ce5dea533dc0dfb4b471c1541e963e6cd5de9b2869135441d not found: ID does not exist" containerID="81bf5aeab9607d8ce5dea533dc0dfb4b471c1541e963e6cd5de9b2869135441d"
Sep 11 11:35:53.644152 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.644105    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81bf5aeab9607d8ce5dea533dc0dfb4b471c1541e963e6cd5de9b2869135441d"} err="failed to get container status \"81bf5aeab9607d8ce5dea533dc0dfb4b471c1541e963e6cd5de9b2869135441d\": rpc error: code = NotFound desc = could not find container \"81bf5aeab9607d8ce5dea533dc0dfb4b471c1541e963e6cd5de9b2869135441d\": container with ID starting with 81bf5aeab9607d8ce5dea533dc0dfb4b471c1541e963e6cd5de9b2869135441d not found: ID does not exist"
Sep 11 11:35:53.654900 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.654874    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65f55d9874-p6xzw"]
Sep 11 11:35:53.658613 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:53.658593    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-65f55d9874-p6xzw"]
Sep 11 11:35:54.081007 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:54.080976    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d08cc13-d976-4959-8fc7-a851d6d4c82d" path="/var/lib/kubelet/pods/5d08cc13-d976-4959-8fc7-a851d6d4c82d/volumes"
Sep 11 11:35:55.053303 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:55.053268    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:35:55.153040 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:35:55.153005    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:36:00.778589 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:00.778527    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-68f64d9c64-w89kc" podUID="abb23749-d480-4ef5-94a8-ab11e3a86cc7" containerName="console" containerID="cri-o://12d4199ae80f40cddc16cf5bcbdae3f97c2db4a719187d46ee92fb144b31179e" gracePeriod=15
Sep 11 11:36:01.026253 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.026228    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:36:01.102890 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.102816    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-oauth-serving-cert\") pod \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") "
Sep 11 11:36:01.102890 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.102860    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-service-ca\") pod \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") "
Sep 11 11:36:01.103080 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.102907    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-console-config\") pod \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") "
Sep 11 11:36:01.103080 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.102926    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-trusted-ca-bundle\") pod \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") "
Sep 11 11:36:01.103080 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.102959    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/abb23749-d480-4ef5-94a8-ab11e3a86cc7-console-oauth-config\") pod \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") "
Sep 11 11:36:01.103080 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.102995    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/abb23749-d480-4ef5-94a8-ab11e3a86cc7-console-serving-cert\") pod \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") "
Sep 11 11:36:01.103261 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.103156    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvg4n\" (UniqueName: \"kubernetes.io/projected/abb23749-d480-4ef5-94a8-ab11e3a86cc7-kube-api-access-tvg4n\") pod \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\" (UID: \"abb23749-d480-4ef5-94a8-ab11e3a86cc7\") "
Sep 11 11:36:01.103313 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.103287    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "abb23749-d480-4ef5-94a8-ab11e3a86cc7" (UID: "abb23749-d480-4ef5-94a8-ab11e3a86cc7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:36:01.103393 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.103367    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-console-config" (OuterVolumeSpecName: "console-config") pod "abb23749-d480-4ef5-94a8-ab11e3a86cc7" (UID: "abb23749-d480-4ef5-94a8-ab11e3a86cc7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:36:01.103460 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.103404    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "abb23749-d480-4ef5-94a8-ab11e3a86cc7" (UID: "abb23749-d480-4ef5-94a8-ab11e3a86cc7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:36:01.103529 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.103504    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-service-ca" (OuterVolumeSpecName: "service-ca") pod "abb23749-d480-4ef5-94a8-ab11e3a86cc7" (UID: "abb23749-d480-4ef5-94a8-ab11e3a86cc7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:36:01.103582 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.103528    2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-oauth-serving-cert\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:01.103582 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.103542    2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-console-config\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:01.103582 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.103552    2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-trusted-ca-bundle\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:01.105364 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.105335    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abb23749-d480-4ef5-94a8-ab11e3a86cc7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "abb23749-d480-4ef5-94a8-ab11e3a86cc7" (UID: "abb23749-d480-4ef5-94a8-ab11e3a86cc7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:36:01.105485 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.105408    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abb23749-d480-4ef5-94a8-ab11e3a86cc7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "abb23749-d480-4ef5-94a8-ab11e3a86cc7" (UID: "abb23749-d480-4ef5-94a8-ab11e3a86cc7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:36:01.105548 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.105510    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abb23749-d480-4ef5-94a8-ab11e3a86cc7-kube-api-access-tvg4n" (OuterVolumeSpecName: "kube-api-access-tvg4n") pod "abb23749-d480-4ef5-94a8-ab11e3a86cc7" (UID: "abb23749-d480-4ef5-94a8-ab11e3a86cc7"). InnerVolumeSpecName "kube-api-access-tvg4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:36:01.204622 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.204578    2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/abb23749-d480-4ef5-94a8-ab11e3a86cc7-service-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:01.204622 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.204616    2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/abb23749-d480-4ef5-94a8-ab11e3a86cc7-console-oauth-config\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:01.204622 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.204626    2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/abb23749-d480-4ef5-94a8-ab11e3a86cc7-console-serving-cert\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:01.204622 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.204658    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvg4n\" (UniqueName: \"kubernetes.io/projected/abb23749-d480-4ef5-94a8-ab11e3a86cc7-kube-api-access-tvg4n\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:01.668807 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.668768    2575 generic.go:356] "Generic (PLEG): container finished" podID="abb23749-d480-4ef5-94a8-ab11e3a86cc7" containerID="12d4199ae80f40cddc16cf5bcbdae3f97c2db4a719187d46ee92fb144b31179e" exitCode=0
Sep 11 11:36:01.668807 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.668812    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68f64d9c64-w89kc"
Sep 11 11:36:01.669021 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.668852    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68f64d9c64-w89kc" event={"ID":"abb23749-d480-4ef5-94a8-ab11e3a86cc7","Type":"ContainerDied","Data":"12d4199ae80f40cddc16cf5bcbdae3f97c2db4a719187d46ee92fb144b31179e"}
Sep 11 11:36:01.669021 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.668902    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68f64d9c64-w89kc" event={"ID":"abb23749-d480-4ef5-94a8-ab11e3a86cc7","Type":"ContainerDied","Data":"2e026a8a7024ca66fc73696341fa618c5262719a402f70178e7beba10799693f"}
Sep 11 11:36:01.669021 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.668926    2575 scope.go:117] "RemoveContainer" containerID="12d4199ae80f40cddc16cf5bcbdae3f97c2db4a719187d46ee92fb144b31179e"
Sep 11 11:36:01.678077 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.678061    2575 scope.go:117] "RemoveContainer" containerID="12d4199ae80f40cddc16cf5bcbdae3f97c2db4a719187d46ee92fb144b31179e"
Sep 11 11:36:01.678351 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:36:01.678335    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12d4199ae80f40cddc16cf5bcbdae3f97c2db4a719187d46ee92fb144b31179e\": container with ID starting with 12d4199ae80f40cddc16cf5bcbdae3f97c2db4a719187d46ee92fb144b31179e not found: ID does not exist" containerID="12d4199ae80f40cddc16cf5bcbdae3f97c2db4a719187d46ee92fb144b31179e"
Sep 11 11:36:01.678405 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.678359    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12d4199ae80f40cddc16cf5bcbdae3f97c2db4a719187d46ee92fb144b31179e"} err="failed to get container status \"12d4199ae80f40cddc16cf5bcbdae3f97c2db4a719187d46ee92fb144b31179e\": rpc error: code = NotFound desc = could not find container \"12d4199ae80f40cddc16cf5bcbdae3f97c2db4a719187d46ee92fb144b31179e\": container with ID starting with 12d4199ae80f40cddc16cf5bcbdae3f97c2db4a719187d46ee92fb144b31179e not found: ID does not exist"
Sep 11 11:36:01.690075 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.690053    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68f64d9c64-w89kc"]
Sep 11 11:36:01.696001 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:01.695981    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68f64d9c64-w89kc"]
Sep 11 11:36:02.084126 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:02.084082    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abb23749-d480-4ef5-94a8-ab11e3a86cc7" path="/var/lib/kubelet/pods/abb23749-d480-4ef5-94a8-ab11e3a86cc7/volumes"
Sep 11 11:36:05.526133 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:05.526098    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wklxl"
Sep 11 11:36:07.562659 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:07.562614    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:36:07.566410 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:07.566385    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7c76d5974f-5ntt7"
Sep 11 11:36:11.719497 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:11.719456    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c7c544647-2qvfm" podUID="680cef74-e325-4d0a-b6a4-366cdd8f6a58" containerName="console" containerID="cri-o://9d978a5a48996e6f00a1bfe079793608c7bb5b8e9e0031cf1a249770fbed3185" gracePeriod=15
Sep 11 11:36:12.013600 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.013558    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:36:12.102373 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.102344    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/680cef74-e325-4d0a-b6a4-366cdd8f6a58-console-serving-cert\") pod \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") "
Sep 11 11:36:12.102540 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.102389    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/680cef74-e325-4d0a-b6a4-366cdd8f6a58-service-ca\") pod \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") "
Sep 11 11:36:12.102540 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.102409    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/680cef74-e325-4d0a-b6a4-366cdd8f6a58-console-config\") pod \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") "
Sep 11 11:36:12.102680 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.102604    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/680cef74-e325-4d0a-b6a4-366cdd8f6a58-oauth-serving-cert\") pod \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") "
Sep 11 11:36:12.102740 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.102703    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62vtd\" (UniqueName: \"kubernetes.io/projected/680cef74-e325-4d0a-b6a4-366cdd8f6a58-kube-api-access-62vtd\") pod \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") "
Sep 11 11:36:12.102740 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.102735    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/680cef74-e325-4d0a-b6a4-366cdd8f6a58-console-oauth-config\") pod \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\" (UID: \"680cef74-e325-4d0a-b6a4-366cdd8f6a58\") "
Sep 11 11:36:12.102849 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.102781    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/680cef74-e325-4d0a-b6a4-366cdd8f6a58-service-ca" (OuterVolumeSpecName: "service-ca") pod "680cef74-e325-4d0a-b6a4-366cdd8f6a58" (UID: "680cef74-e325-4d0a-b6a4-366cdd8f6a58"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:36:12.102849 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.102790    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/680cef74-e325-4d0a-b6a4-366cdd8f6a58-console-config" (OuterVolumeSpecName: "console-config") pod "680cef74-e325-4d0a-b6a4-366cdd8f6a58" (UID: "680cef74-e325-4d0a-b6a4-366cdd8f6a58"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:36:12.103030 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.102992    2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/680cef74-e325-4d0a-b6a4-366cdd8f6a58-service-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:12.103030 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.103018    2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/680cef74-e325-4d0a-b6a4-366cdd8f6a58-console-config\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:12.103030 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.102995    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/680cef74-e325-4d0a-b6a4-366cdd8f6a58-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "680cef74-e325-4d0a-b6a4-366cdd8f6a58" (UID: "680cef74-e325-4d0a-b6a4-366cdd8f6a58"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:36:12.104748 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.104729    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/680cef74-e325-4d0a-b6a4-366cdd8f6a58-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "680cef74-e325-4d0a-b6a4-366cdd8f6a58" (UID: "680cef74-e325-4d0a-b6a4-366cdd8f6a58"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:36:12.104898 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.104879    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/680cef74-e325-4d0a-b6a4-366cdd8f6a58-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "680cef74-e325-4d0a-b6a4-366cdd8f6a58" (UID: "680cef74-e325-4d0a-b6a4-366cdd8f6a58"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:36:12.104985 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.104934    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/680cef74-e325-4d0a-b6a4-366cdd8f6a58-kube-api-access-62vtd" (OuterVolumeSpecName: "kube-api-access-62vtd") pod "680cef74-e325-4d0a-b6a4-366cdd8f6a58" (UID: "680cef74-e325-4d0a-b6a4-366cdd8f6a58"). InnerVolumeSpecName "kube-api-access-62vtd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:36:12.204522 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.204481    2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/680cef74-e325-4d0a-b6a4-366cdd8f6a58-oauth-serving-cert\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:12.204522 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.204511    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62vtd\" (UniqueName: \"kubernetes.io/projected/680cef74-e325-4d0a-b6a4-366cdd8f6a58-kube-api-access-62vtd\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:12.204522 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.204522    2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/680cef74-e325-4d0a-b6a4-366cdd8f6a58-console-oauth-config\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:12.204522 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.204530    2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/680cef74-e325-4d0a-b6a4-366cdd8f6a58-console-serving-cert\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:12.714846 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.714812    2575 generic.go:356] "Generic (PLEG): container finished" podID="680cef74-e325-4d0a-b6a4-366cdd8f6a58" containerID="9d978a5a48996e6f00a1bfe079793608c7bb5b8e9e0031cf1a249770fbed3185" exitCode=0
Sep 11 11:36:12.715053 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.714863    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c7c544647-2qvfm"
Sep 11 11:36:12.715053 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.714897    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c7c544647-2qvfm" event={"ID":"680cef74-e325-4d0a-b6a4-366cdd8f6a58","Type":"ContainerDied","Data":"9d978a5a48996e6f00a1bfe079793608c7bb5b8e9e0031cf1a249770fbed3185"}
Sep 11 11:36:12.715053 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.714949    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c7c544647-2qvfm" event={"ID":"680cef74-e325-4d0a-b6a4-366cdd8f6a58","Type":"ContainerDied","Data":"f7ae265f4dc2e04ea54d76ccf17803cb86a8ef802d0b7e66e043ebf4ab35a10b"}
Sep 11 11:36:12.715053 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.714970    2575 scope.go:117] "RemoveContainer" containerID="9d978a5a48996e6f00a1bfe079793608c7bb5b8e9e0031cf1a249770fbed3185"
Sep 11 11:36:12.723960 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.723704    2575 scope.go:117] "RemoveContainer" containerID="9d978a5a48996e6f00a1bfe079793608c7bb5b8e9e0031cf1a249770fbed3185"
Sep 11 11:36:12.724187 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:36:12.723984    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d978a5a48996e6f00a1bfe079793608c7bb5b8e9e0031cf1a249770fbed3185\": container with ID starting with 9d978a5a48996e6f00a1bfe079793608c7bb5b8e9e0031cf1a249770fbed3185 not found: ID does not exist" containerID="9d978a5a48996e6f00a1bfe079793608c7bb5b8e9e0031cf1a249770fbed3185"
Sep 11 11:36:12.724187 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.724006    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d978a5a48996e6f00a1bfe079793608c7bb5b8e9e0031cf1a249770fbed3185"} err="failed to get container status \"9d978a5a48996e6f00a1bfe079793608c7bb5b8e9e0031cf1a249770fbed3185\": rpc error: code = NotFound desc = could not find container \"9d978a5a48996e6f00a1bfe079793608c7bb5b8e9e0031cf1a249770fbed3185\": container with ID starting with 9d978a5a48996e6f00a1bfe079793608c7bb5b8e9e0031cf1a249770fbed3185 not found: ID does not exist"
Sep 11 11:36:12.735694 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.735670    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c7c544647-2qvfm"]
Sep 11 11:36:12.738801 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:12.738771    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c7c544647-2qvfm"]
Sep 11 11:36:14.081398 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:14.081368    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="680cef74-e325-4d0a-b6a4-366cdd8f6a58" path="/var/lib/kubelet/pods/680cef74-e325-4d0a-b6a4-366cdd8f6a58/volumes"
Sep 11 11:36:19.504931 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:19.504894    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:36:19.578454 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:19.578423    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:36:21.391249 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:21.391215    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:36:29.230788 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:29.230753    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:29.250825 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:29.250800    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:29.808618 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:29.808593    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:38.638305 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:38.638266    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:36:38.704885 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:38.704853    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:36:39.072055 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:39.072023    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:36:39.134875 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:39.134847    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:36:42.905838 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:42.905803    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:36:42.906250 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:42.906085    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="alertmanager" containerID="cri-o://f502523aae764264ec2c062e09223b2a9d68fa3f7e271ae78b47606a49eed818" gracePeriod=120
Sep 11 11:36:42.906250 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:42.906154    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="kube-rbac-proxy-metric" containerID="cri-o://028afa777b7d5e590ec4e9d2a6679183aa7acdaefaa2e786c6dae28492f77176" gracePeriod=120
Sep 11 11:36:42.906380 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:42.906239    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="kube-rbac-proxy-web" containerID="cri-o://0b29a4308bfe45d5be18034a89c220d246d44192830fd02183e95072379d66db" gracePeriod=120
Sep 11 11:36:42.906380 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:42.906270    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="config-reloader" containerID="cri-o://f4b182b56923c0468e8dd68094e80d747e0a40192f51f28e153a016dc8720b3a" gracePeriod=120
Sep 11 11:36:42.906380 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:42.906341    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="kube-rbac-proxy" containerID="cri-o://41a1d65662827f974b103a9438ed149dac3f326847a814b259607a2962310f9a" gracePeriod=120
Sep 11 11:36:42.906380 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:42.906299    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="prom-label-proxy" containerID="cri-o://adfaf34b6db0b7c7fa79ffabafca31d3828ee038f7462d45d4bf658d9941e723" gracePeriod=120
Sep 11 11:36:43.076704 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.076671    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b44699f76-7pcsz"]
Sep 11 11:36:43.077022 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.077010    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="680cef74-e325-4d0a-b6a4-366cdd8f6a58" containerName="console"
Sep 11 11:36:43.077061 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.077024    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="680cef74-e325-4d0a-b6a4-366cdd8f6a58" containerName="console"
Sep 11 11:36:43.077061 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.077035    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abb23749-d480-4ef5-94a8-ab11e3a86cc7" containerName="console"
Sep 11 11:36:43.077061 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.077041    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="abb23749-d480-4ef5-94a8-ab11e3a86cc7" containerName="console"
Sep 11 11:36:43.077146 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.077059    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d08cc13-d976-4959-8fc7-a851d6d4c82d" containerName="console"
Sep 11 11:36:43.077146 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.077070    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d08cc13-d976-4959-8fc7-a851d6d4c82d" containerName="console"
Sep 11 11:36:43.077146 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.077127    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="680cef74-e325-4d0a-b6a4-366cdd8f6a58" containerName="console"
Sep 11 11:36:43.077146 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.077137    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="abb23749-d480-4ef5-94a8-ab11e3a86cc7" containerName="console"
Sep 11 11:36:43.077264 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.077146    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d08cc13-d976-4959-8fc7-a851d6d4c82d" containerName="console"
Sep 11 11:36:43.079002 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.078985    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:43.089792 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.089768    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b44699f76-7pcsz"]
Sep 11 11:36:43.143590 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.143557    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:36:43.169707 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.168704    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe149968-7812-4489-960d-c070b29e300b-console-serving-cert\") pod \"console-b44699f76-7pcsz\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") " pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:43.169707 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.168772    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfxzh\" (UniqueName: \"kubernetes.io/projected/fe149968-7812-4489-960d-c070b29e300b-kube-api-access-jfxzh\") pod \"console-b44699f76-7pcsz\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") " pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:43.169707 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.168806    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe149968-7812-4489-960d-c070b29e300b-console-oauth-config\") pod \"console-b44699f76-7pcsz\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") " pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:43.169707 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.168846    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-console-config\") pod \"console-b44699f76-7pcsz\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") " pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:43.169707 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.168879    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-oauth-serving-cert\") pod \"console-b44699f76-7pcsz\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") " pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:43.169707 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.168922    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-trusted-ca-bundle\") pod \"console-b44699f76-7pcsz\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") " pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:43.169707 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.168972    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-service-ca\") pod \"console-b44699f76-7pcsz\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") " pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:43.212676 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.212608    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:36:43.269943 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.269907    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe149968-7812-4489-960d-c070b29e300b-console-oauth-config\") pod \"console-b44699f76-7pcsz\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") " pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:43.270134 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.269961    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-console-config\") pod \"console-b44699f76-7pcsz\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") " pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:43.270134 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.269990    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-oauth-serving-cert\") pod \"console-b44699f76-7pcsz\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") " pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:43.270134 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.270027    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-trusted-ca-bundle\") pod \"console-b44699f76-7pcsz\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") " pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:43.270134 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.270067    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-service-ca\") pod \"console-b44699f76-7pcsz\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") " pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:43.270134 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.270101    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe149968-7812-4489-960d-c070b29e300b-console-serving-cert\") pod \"console-b44699f76-7pcsz\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") " pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:43.270372 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.270134    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfxzh\" (UniqueName: \"kubernetes.io/projected/fe149968-7812-4489-960d-c070b29e300b-kube-api-access-jfxzh\") pod \"console-b44699f76-7pcsz\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") " pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:43.271006 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.270979    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-console-config\") pod \"console-b44699f76-7pcsz\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") " pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:43.271119 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.271058    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-oauth-serving-cert\") pod \"console-b44699f76-7pcsz\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") " pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:43.271119 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.271099    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-service-ca\") pod \"console-b44699f76-7pcsz\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") " pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:43.271219 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.271117    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-trusted-ca-bundle\") pod \"console-b44699f76-7pcsz\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") " pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:43.272750 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.272688    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe149968-7812-4489-960d-c070b29e300b-console-oauth-config\") pod \"console-b44699f76-7pcsz\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") " pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:43.273406 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.273381    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe149968-7812-4489-960d-c070b29e300b-console-serving-cert\") pod \"console-b44699f76-7pcsz\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") " pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:43.280289 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.280267    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfxzh\" (UniqueName: \"kubernetes.io/projected/fe149968-7812-4489-960d-c070b29e300b-kube-api-access-jfxzh\") pod \"console-b44699f76-7pcsz\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") " pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:43.387807 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.387775    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:43.658108 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.658084    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b44699f76-7pcsz"]
Sep 11 11:36:43.660341 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:36:43.660316    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe149968_7812_4489_960d_c070b29e300b.slice/crio-bc8ca6aa3c28e43dd883695f1f2155c443707bce1a90ef55a7d91b7b88f99491 WatchSource:0}: Error finding container bc8ca6aa3c28e43dd883695f1f2155c443707bce1a90ef55a7d91b7b88f99491: Status 404 returned error can't find the container with id bc8ca6aa3c28e43dd883695f1f2155c443707bce1a90ef55a7d91b7b88f99491
Sep 11 11:36:43.847886 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.847858    2575 generic.go:356] "Generic (PLEG): container finished" podID="bbe42078-01c1-49d6-a4df-785832fb726e" containerID="adfaf34b6db0b7c7fa79ffabafca31d3828ee038f7462d45d4bf658d9941e723" exitCode=0
Sep 11 11:36:43.847886 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.847880    2575 generic.go:356] "Generic (PLEG): container finished" podID="bbe42078-01c1-49d6-a4df-785832fb726e" containerID="028afa777b7d5e590ec4e9d2a6679183aa7acdaefaa2e786c6dae28492f77176" exitCode=0
Sep 11 11:36:43.847886 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.847886    2575 generic.go:356] "Generic (PLEG): container finished" podID="bbe42078-01c1-49d6-a4df-785832fb726e" containerID="41a1d65662827f974b103a9438ed149dac3f326847a814b259607a2962310f9a" exitCode=0
Sep 11 11:36:43.847886 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.847892    2575 generic.go:356] "Generic (PLEG): container finished" podID="bbe42078-01c1-49d6-a4df-785832fb726e" containerID="f4b182b56923c0468e8dd68094e80d747e0a40192f51f28e153a016dc8720b3a" exitCode=0
Sep 11 11:36:43.848172 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.847899    2575 generic.go:356] "Generic (PLEG): container finished" podID="bbe42078-01c1-49d6-a4df-785832fb726e" containerID="f502523aae764264ec2c062e09223b2a9d68fa3f7e271ae78b47606a49eed818" exitCode=0
Sep 11 11:36:43.848172 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.847937    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bbe42078-01c1-49d6-a4df-785832fb726e","Type":"ContainerDied","Data":"adfaf34b6db0b7c7fa79ffabafca31d3828ee038f7462d45d4bf658d9941e723"}
Sep 11 11:36:43.848172 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.848017    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bbe42078-01c1-49d6-a4df-785832fb726e","Type":"ContainerDied","Data":"028afa777b7d5e590ec4e9d2a6679183aa7acdaefaa2e786c6dae28492f77176"}
Sep 11 11:36:43.848172 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.848047    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bbe42078-01c1-49d6-a4df-785832fb726e","Type":"ContainerDied","Data":"41a1d65662827f974b103a9438ed149dac3f326847a814b259607a2962310f9a"}
Sep 11 11:36:43.848172 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.848070    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bbe42078-01c1-49d6-a4df-785832fb726e","Type":"ContainerDied","Data":"f4b182b56923c0468e8dd68094e80d747e0a40192f51f28e153a016dc8720b3a"}
Sep 11 11:36:43.848172 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.848100    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bbe42078-01c1-49d6-a4df-785832fb726e","Type":"ContainerDied","Data":"f502523aae764264ec2c062e09223b2a9d68fa3f7e271ae78b47606a49eed818"}
Sep 11 11:36:43.849482 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.849458    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b44699f76-7pcsz" event={"ID":"fe149968-7812-4489-960d-c070b29e300b","Type":"ContainerStarted","Data":"7b995bab0567804f9652b18e44109582786f38db59f3b6e310dc5344b3b610e1"}
Sep 11 11:36:43.849603 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.849496    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b44699f76-7pcsz" event={"ID":"fe149968-7812-4489-960d-c070b29e300b","Type":"ContainerStarted","Data":"bc8ca6aa3c28e43dd883695f1f2155c443707bce1a90ef55a7d91b7b88f99491"}
Sep 11 11:36:43.867428 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:43.867381    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b44699f76-7pcsz" podStartSLOduration=0.867363248 podStartE2EDuration="867.363248ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:36:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:36:43.866243503 +0000 UTC m=+144.286849532" watchObservedRunningTime="2026-09-11 11:36:43.867363248 +0000 UTC m=+144.287969278"
Sep 11 11:36:44.152379 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.152355    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:44.177849 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.177829    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jd9l\" (UniqueName: \"kubernetes.io/projected/bbe42078-01c1-49d6-a4df-785832fb726e-kube-api-access-9jd9l\") pod \"bbe42078-01c1-49d6-a4df-785832fb726e\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") "
Sep 11 11:36:44.177961 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.177864    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-web-config\") pod \"bbe42078-01c1-49d6-a4df-785832fb726e\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") "
Sep 11 11:36:44.177961 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.177892    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-cluster-tls-config\") pod \"bbe42078-01c1-49d6-a4df-785832fb726e\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") "
Sep 11 11:36:44.177961 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.177913    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-config-volume\") pod \"bbe42078-01c1-49d6-a4df-785832fb726e\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") "
Sep 11 11:36:44.177961 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.177945    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-kube-rbac-proxy\") pod \"bbe42078-01c1-49d6-a4df-785832fb726e\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") "
Sep 11 11:36:44.178159 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.177969    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-main-tls\") pod \"bbe42078-01c1-49d6-a4df-785832fb726e\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") "
Sep 11 11:36:44.178159 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.178029    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"bbe42078-01c1-49d6-a4df-785832fb726e\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") "
Sep 11 11:36:44.179256 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.178388    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbe42078-01c1-49d6-a4df-785832fb726e-metrics-client-ca\") pod \"bbe42078-01c1-49d6-a4df-785832fb726e\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") "
Sep 11 11:36:44.179256 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.178432    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bbe42078-01c1-49d6-a4df-785832fb726e-config-out\") pod \"bbe42078-01c1-49d6-a4df-785832fb726e\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") "
Sep 11 11:36:44.179256 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.178465    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbe42078-01c1-49d6-a4df-785832fb726e-alertmanager-trusted-ca-bundle\") pod \"bbe42078-01c1-49d6-a4df-785832fb726e\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") "
Sep 11 11:36:44.179256 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.178543    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bbe42078-01c1-49d6-a4df-785832fb726e-tls-assets\") pod \"bbe42078-01c1-49d6-a4df-785832fb726e\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") "
Sep 11 11:36:44.179256 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.178570    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-kube-rbac-proxy-web\") pod \"bbe42078-01c1-49d6-a4df-785832fb726e\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") "
Sep 11 11:36:44.179256 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.178625    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bbe42078-01c1-49d6-a4df-785832fb726e-alertmanager-main-db\") pod \"bbe42078-01c1-49d6-a4df-785832fb726e\" (UID: \"bbe42078-01c1-49d6-a4df-785832fb726e\") "
Sep 11 11:36:44.179256 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.178835    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbe42078-01c1-49d6-a4df-785832fb726e-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "bbe42078-01c1-49d6-a4df-785832fb726e" (UID: "bbe42078-01c1-49d6-a4df-785832fb726e"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:36:44.179256 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.179089    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbe42078-01c1-49d6-a4df-785832fb726e-metrics-client-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:44.181958 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.181465    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbe42078-01c1-49d6-a4df-785832fb726e-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "bbe42078-01c1-49d6-a4df-785832fb726e" (UID: "bbe42078-01c1-49d6-a4df-785832fb726e"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:36:44.182507 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.182375    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "bbe42078-01c1-49d6-a4df-785832fb726e" (UID: "bbe42078-01c1-49d6-a4df-785832fb726e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:36:44.185459 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.183448    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbe42078-01c1-49d6-a4df-785832fb726e-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "bbe42078-01c1-49d6-a4df-785832fb726e" (UID: "bbe42078-01c1-49d6-a4df-785832fb726e"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:36:44.185459 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.184660    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbe42078-01c1-49d6-a4df-785832fb726e-kube-api-access-9jd9l" (OuterVolumeSpecName: "kube-api-access-9jd9l") pod "bbe42078-01c1-49d6-a4df-785832fb726e" (UID: "bbe42078-01c1-49d6-a4df-785832fb726e"). InnerVolumeSpecName "kube-api-access-9jd9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:36:44.185459 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.184732    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbe42078-01c1-49d6-a4df-785832fb726e-config-out" (OuterVolumeSpecName: "config-out") pod "bbe42078-01c1-49d6-a4df-785832fb726e" (UID: "bbe42078-01c1-49d6-a4df-785832fb726e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:36:44.186097 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.185894    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-config-volume" (OuterVolumeSpecName: "config-volume") pod "bbe42078-01c1-49d6-a4df-785832fb726e" (UID: "bbe42078-01c1-49d6-a4df-785832fb726e"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:36:44.186754 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.186726    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "bbe42078-01c1-49d6-a4df-785832fb726e" (UID: "bbe42078-01c1-49d6-a4df-785832fb726e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:36:44.186853 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.186758    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "bbe42078-01c1-49d6-a4df-785832fb726e" (UID: "bbe42078-01c1-49d6-a4df-785832fb726e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:36:44.186919 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.186898    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "bbe42078-01c1-49d6-a4df-785832fb726e" (UID: "bbe42078-01c1-49d6-a4df-785832fb726e"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:36:44.186919 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.186905    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbe42078-01c1-49d6-a4df-785832fb726e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "bbe42078-01c1-49d6-a4df-785832fb726e" (UID: "bbe42078-01c1-49d6-a4df-785832fb726e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:36:44.189423 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.189248    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "bbe42078-01c1-49d6-a4df-785832fb726e" (UID: "bbe42078-01c1-49d6-a4df-785832fb726e"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:36:44.198432 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.198394    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-web-config" (OuterVolumeSpecName: "web-config") pod "bbe42078-01c1-49d6-a4df-785832fb726e" (UID: "bbe42078-01c1-49d6-a4df-785832fb726e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:36:44.280124 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.280092    2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bbe42078-01c1-49d6-a4df-785832fb726e-tls-assets\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:44.280124 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.280118    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:44.280124 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.280128    2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bbe42078-01c1-49d6-a4df-785832fb726e-alertmanager-main-db\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:44.280381 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.280138    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jd9l\" (UniqueName: \"kubernetes.io/projected/bbe42078-01c1-49d6-a4df-785832fb726e-kube-api-access-9jd9l\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:44.280381 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.280146    2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-web-config\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:44.280381 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.280154    2575 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-cluster-tls-config\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:44.280381 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.280161    2575 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-config-volume\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:44.280381 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.280169    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:44.280381 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.280176    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-main-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:44.280381 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.280185    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bbe42078-01c1-49d6-a4df-785832fb726e-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:44.280381 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.280193    2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bbe42078-01c1-49d6-a4df-785832fb726e-config-out\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:44.280381 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.280200    2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbe42078-01c1-49d6-a4df-785832fb726e-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:44.861715 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.861682    2575 generic.go:356] "Generic (PLEG): container finished" podID="bbe42078-01c1-49d6-a4df-785832fb726e" containerID="0b29a4308bfe45d5be18034a89c220d246d44192830fd02183e95072379d66db" exitCode=0
Sep 11 11:36:44.861910 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.861765    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:44.861910 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.861772    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bbe42078-01c1-49d6-a4df-785832fb726e","Type":"ContainerDied","Data":"0b29a4308bfe45d5be18034a89c220d246d44192830fd02183e95072379d66db"}
Sep 11 11:36:44.861910 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.861884    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bbe42078-01c1-49d6-a4df-785832fb726e","Type":"ContainerDied","Data":"1c8fedaba5e3786609ef657d74b90785c4eeb85e1622261d4bf0505c7ba42c3e"}
Sep 11 11:36:44.862072 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.861932    2575 scope.go:117] "RemoveContainer" containerID="adfaf34b6db0b7c7fa79ffabafca31d3828ee038f7462d45d4bf658d9941e723"
Sep 11 11:36:44.869618 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.869598    2575 scope.go:117] "RemoveContainer" containerID="028afa777b7d5e590ec4e9d2a6679183aa7acdaefaa2e786c6dae28492f77176"
Sep 11 11:36:44.877140 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.877125    2575 scope.go:117] "RemoveContainer" containerID="41a1d65662827f974b103a9438ed149dac3f326847a814b259607a2962310f9a"
Sep 11 11:36:44.884199 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.884182    2575 scope.go:117] "RemoveContainer" containerID="0b29a4308bfe45d5be18034a89c220d246d44192830fd02183e95072379d66db"
Sep 11 11:36:44.888823 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.888801    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:36:44.892006 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.891991    2575 scope.go:117] "RemoveContainer" containerID="f4b182b56923c0468e8dd68094e80d747e0a40192f51f28e153a016dc8720b3a"
Sep 11 11:36:44.895194 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.895175    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:36:44.899881 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.899859    2575 scope.go:117] "RemoveContainer" containerID="f502523aae764264ec2c062e09223b2a9d68fa3f7e271ae78b47606a49eed818"
Sep 11 11:36:44.907788 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.907763    2575 scope.go:117] "RemoveContainer" containerID="d114e504416e82c519031f0d9c3fa2722d9a1799701fd6d042e9f17b71f1e793"
Sep 11 11:36:44.915432 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.915416    2575 scope.go:117] "RemoveContainer" containerID="adfaf34b6db0b7c7fa79ffabafca31d3828ee038f7462d45d4bf658d9941e723"
Sep 11 11:36:44.915729 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:36:44.915708    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adfaf34b6db0b7c7fa79ffabafca31d3828ee038f7462d45d4bf658d9941e723\": container with ID starting with adfaf34b6db0b7c7fa79ffabafca31d3828ee038f7462d45d4bf658d9941e723 not found: ID does not exist" containerID="adfaf34b6db0b7c7fa79ffabafca31d3828ee038f7462d45d4bf658d9941e723"
Sep 11 11:36:44.915807 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.915737    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adfaf34b6db0b7c7fa79ffabafca31d3828ee038f7462d45d4bf658d9941e723"} err="failed to get container status \"adfaf34b6db0b7c7fa79ffabafca31d3828ee038f7462d45d4bf658d9941e723\": rpc error: code = NotFound desc = could not find container \"adfaf34b6db0b7c7fa79ffabafca31d3828ee038f7462d45d4bf658d9941e723\": container with ID starting with adfaf34b6db0b7c7fa79ffabafca31d3828ee038f7462d45d4bf658d9941e723 not found: ID does not exist"
Sep 11 11:36:44.915807 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.915757    2575 scope.go:117] "RemoveContainer" containerID="028afa777b7d5e590ec4e9d2a6679183aa7acdaefaa2e786c6dae28492f77176"
Sep 11 11:36:44.916062 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:36:44.916027    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"028afa777b7d5e590ec4e9d2a6679183aa7acdaefaa2e786c6dae28492f77176\": container with ID starting with 028afa777b7d5e590ec4e9d2a6679183aa7acdaefaa2e786c6dae28492f77176 not found: ID does not exist" containerID="028afa777b7d5e590ec4e9d2a6679183aa7acdaefaa2e786c6dae28492f77176"
Sep 11 11:36:44.916137 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.916072    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"028afa777b7d5e590ec4e9d2a6679183aa7acdaefaa2e786c6dae28492f77176"} err="failed to get container status \"028afa777b7d5e590ec4e9d2a6679183aa7acdaefaa2e786c6dae28492f77176\": rpc error: code = NotFound desc = could not find container \"028afa777b7d5e590ec4e9d2a6679183aa7acdaefaa2e786c6dae28492f77176\": container with ID starting with 028afa777b7d5e590ec4e9d2a6679183aa7acdaefaa2e786c6dae28492f77176 not found: ID does not exist"
Sep 11 11:36:44.916137 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.916095    2575 scope.go:117] "RemoveContainer" containerID="41a1d65662827f974b103a9438ed149dac3f326847a814b259607a2962310f9a"
Sep 11 11:36:44.916370 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:36:44.916353    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41a1d65662827f974b103a9438ed149dac3f326847a814b259607a2962310f9a\": container with ID starting with 41a1d65662827f974b103a9438ed149dac3f326847a814b259607a2962310f9a not found: ID does not exist" containerID="41a1d65662827f974b103a9438ed149dac3f326847a814b259607a2962310f9a"
Sep 11 11:36:44.916420 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.916376    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41a1d65662827f974b103a9438ed149dac3f326847a814b259607a2962310f9a"} err="failed to get container status \"41a1d65662827f974b103a9438ed149dac3f326847a814b259607a2962310f9a\": rpc error: code = NotFound desc = could not find container \"41a1d65662827f974b103a9438ed149dac3f326847a814b259607a2962310f9a\": container with ID starting with 41a1d65662827f974b103a9438ed149dac3f326847a814b259607a2962310f9a not found: ID does not exist"
Sep 11 11:36:44.916420 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.916390    2575 scope.go:117] "RemoveContainer" containerID="0b29a4308bfe45d5be18034a89c220d246d44192830fd02183e95072379d66db"
Sep 11 11:36:44.916604 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:36:44.916587    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b29a4308bfe45d5be18034a89c220d246d44192830fd02183e95072379d66db\": container with ID starting with 0b29a4308bfe45d5be18034a89c220d246d44192830fd02183e95072379d66db not found: ID does not exist" containerID="0b29a4308bfe45d5be18034a89c220d246d44192830fd02183e95072379d66db"
Sep 11 11:36:44.916736 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.916612    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b29a4308bfe45d5be18034a89c220d246d44192830fd02183e95072379d66db"} err="failed to get container status \"0b29a4308bfe45d5be18034a89c220d246d44192830fd02183e95072379d66db\": rpc error: code = NotFound desc = could not find container \"0b29a4308bfe45d5be18034a89c220d246d44192830fd02183e95072379d66db\": container with ID starting with 0b29a4308bfe45d5be18034a89c220d246d44192830fd02183e95072379d66db not found: ID does not exist"
Sep 11 11:36:44.916736 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.916673    2575 scope.go:117] "RemoveContainer" containerID="f4b182b56923c0468e8dd68094e80d747e0a40192f51f28e153a016dc8720b3a"
Sep 11 11:36:44.916914 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:36:44.916898    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4b182b56923c0468e8dd68094e80d747e0a40192f51f28e153a016dc8720b3a\": container with ID starting with f4b182b56923c0468e8dd68094e80d747e0a40192f51f28e153a016dc8720b3a not found: ID does not exist" containerID="f4b182b56923c0468e8dd68094e80d747e0a40192f51f28e153a016dc8720b3a"
Sep 11 11:36:44.916954 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.916919    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4b182b56923c0468e8dd68094e80d747e0a40192f51f28e153a016dc8720b3a"} err="failed to get container status \"f4b182b56923c0468e8dd68094e80d747e0a40192f51f28e153a016dc8720b3a\": rpc error: code = NotFound desc = could not find container \"f4b182b56923c0468e8dd68094e80d747e0a40192f51f28e153a016dc8720b3a\": container with ID starting with f4b182b56923c0468e8dd68094e80d747e0a40192f51f28e153a016dc8720b3a not found: ID does not exist"
Sep 11 11:36:44.916954 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.916932    2575 scope.go:117] "RemoveContainer" containerID="f502523aae764264ec2c062e09223b2a9d68fa3f7e271ae78b47606a49eed818"
Sep 11 11:36:44.917202 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:36:44.917187    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f502523aae764264ec2c062e09223b2a9d68fa3f7e271ae78b47606a49eed818\": container with ID starting with f502523aae764264ec2c062e09223b2a9d68fa3f7e271ae78b47606a49eed818 not found: ID does not exist" containerID="f502523aae764264ec2c062e09223b2a9d68fa3f7e271ae78b47606a49eed818"
Sep 11 11:36:44.917260 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.917202    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f502523aae764264ec2c062e09223b2a9d68fa3f7e271ae78b47606a49eed818"} err="failed to get container status \"f502523aae764264ec2c062e09223b2a9d68fa3f7e271ae78b47606a49eed818\": rpc error: code = NotFound desc = could not find container \"f502523aae764264ec2c062e09223b2a9d68fa3f7e271ae78b47606a49eed818\": container with ID starting with f502523aae764264ec2c062e09223b2a9d68fa3f7e271ae78b47606a49eed818 not found: ID does not exist"
Sep 11 11:36:44.917260 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.917213    2575 scope.go:117] "RemoveContainer" containerID="d114e504416e82c519031f0d9c3fa2722d9a1799701fd6d042e9f17b71f1e793"
Sep 11 11:36:44.917416 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:36:44.917400    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d114e504416e82c519031f0d9c3fa2722d9a1799701fd6d042e9f17b71f1e793\": container with ID starting with d114e504416e82c519031f0d9c3fa2722d9a1799701fd6d042e9f17b71f1e793 not found: ID does not exist" containerID="d114e504416e82c519031f0d9c3fa2722d9a1799701fd6d042e9f17b71f1e793"
Sep 11 11:36:44.917456 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.917420    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d114e504416e82c519031f0d9c3fa2722d9a1799701fd6d042e9f17b71f1e793"} err="failed to get container status \"d114e504416e82c519031f0d9c3fa2722d9a1799701fd6d042e9f17b71f1e793\": rpc error: code = NotFound desc = could not find container \"d114e504416e82c519031f0d9c3fa2722d9a1799701fd6d042e9f17b71f1e793\": container with ID starting with d114e504416e82c519031f0d9c3fa2722d9a1799701fd6d042e9f17b71f1e793 not found: ID does not exist"
Sep 11 11:36:44.923838 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.923818    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:36:44.924134 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.924123    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="kube-rbac-proxy"
Sep 11 11:36:44.924179 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.924136    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="kube-rbac-proxy"
Sep 11 11:36:44.924179 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.924144    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="alertmanager"
Sep 11 11:36:44.924179 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.924150    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="alertmanager"
Sep 11 11:36:44.924179 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.924159    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="init-config-reloader"
Sep 11 11:36:44.924179 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.924164    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="init-config-reloader"
Sep 11 11:36:44.924179 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.924178    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="config-reloader"
Sep 11 11:36:44.924349 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.924184    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="config-reloader"
Sep 11 11:36:44.924349 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.924191    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="prom-label-proxy"
Sep 11 11:36:44.924349 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.924196    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="prom-label-proxy"
Sep 11 11:36:44.924349 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.924204    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="kube-rbac-proxy-metric"
Sep 11 11:36:44.924349 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.924209    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="kube-rbac-proxy-metric"
Sep 11 11:36:44.924349 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.924216    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="kube-rbac-proxy-web"
Sep 11 11:36:44.924349 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.924222    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="kube-rbac-proxy-web"
Sep 11 11:36:44.924349 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.924267    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="config-reloader"
Sep 11 11:36:44.924349 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.924274    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="alertmanager"
Sep 11 11:36:44.924349 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.924281    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="prom-label-proxy"
Sep 11 11:36:44.924349 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.924289    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="kube-rbac-proxy"
Sep 11 11:36:44.924349 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.924293    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="kube-rbac-proxy-metric"
Sep 11 11:36:44.924349 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.924300    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" containerName="kube-rbac-proxy-web"
Sep 11 11:36:44.928729 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.928694    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:44.932953 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.932932    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 11 11:36:44.932953 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.932949    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 11 11:36:44.933992 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.933962    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 11 11:36:44.934128 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.934111    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 11 11:36:44.934189 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.934114    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 11 11:36:44.934235 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.934201    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 11 11:36:44.934732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.934696    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 11 11:36:44.934732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.934718    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 11 11:36:44.935429 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.935404    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-l2r4q\""
Sep 11 11:36:44.939249 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.939233    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 11 11:36:44.944658 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.944624    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:36:44.985843 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.985811    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-config-out\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:44.985843 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.985844    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:44.986053 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.985879    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:44.986053 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.985959    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:44.986053 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.986011    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:44.986164 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.986055    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:44.986164 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.986081    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:44.986164 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.986101    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:44.986164 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.986156    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-web-config\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:44.986314 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.986177    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-config-volume\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:44.986314 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.986196    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:44.986314 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.986217    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:44.986314 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:44.986231    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4pq9\" (UniqueName: \"kubernetes.io/projected/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-kube-api-access-s4pq9\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.087131 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.087089    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.087322 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.087140    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.087322 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.087169    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.087322 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.087192    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.087322 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.087217    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.087322 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.087242    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.087322 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.087289    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-web-config\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.087322 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.087316    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-config-volume\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.087712 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.087345    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.087712 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.087380    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.087712 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.087406    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4pq9\" (UniqueName: \"kubernetes.io/projected/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-kube-api-access-s4pq9\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.087712 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.087439    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-config-out\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.087712 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.087465    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.087975 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.087713    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.088303 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.088185    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.088399 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.088325    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.090534 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.090498    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.090677 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.090596    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-web-config\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.090784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.090761    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.090887 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.090863    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-config-out\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.091029 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.091009    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.091195 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.091178    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.091496 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.091481    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.091537 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.091508    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-config-volume\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.092593 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.092577    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.096145 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.096125    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4pq9\" (UniqueName: \"kubernetes.io/projected/1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf-kube-api-access-s4pq9\") pod \"alertmanager-main-0\" (UID: \"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.238115 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.238087    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 11:36:45.516677 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.516650    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 11:36:45.517563 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:36:45.517539    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b1d253b_c33f_4fa5_a26a_0f4d66bb72cf.slice/crio-8254d16ccec979fbcd47850623d76c6670ffbfe7dca8443e4baa66fd56f3b538 WatchSource:0}: Error finding container 8254d16ccec979fbcd47850623d76c6670ffbfe7dca8443e4baa66fd56f3b538: Status 404 returned error can't find the container with id 8254d16ccec979fbcd47850623d76c6670ffbfe7dca8443e4baa66fd56f3b538
Sep 11 11:36:45.868131 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.868044    2575 generic.go:356] "Generic (PLEG): container finished" podID="1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf" containerID="75185e009214b41b1c0959d1dffd87d30e61a4f5971d41c90685f20c838ecae0" exitCode=0
Sep 11 11:36:45.868300 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.868142    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf","Type":"ContainerDied","Data":"75185e009214b41b1c0959d1dffd87d30e61a4f5971d41c90685f20c838ecae0"}
Sep 11 11:36:45.868300 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:45.868223    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf","Type":"ContainerStarted","Data":"8254d16ccec979fbcd47850623d76c6670ffbfe7dca8443e4baa66fd56f3b538"}
Sep 11 11:36:46.085060 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:46.085030    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbe42078-01c1-49d6-a4df-785832fb726e" path="/var/lib/kubelet/pods/bbe42078-01c1-49d6-a4df-785832fb726e/volumes"
Sep 11 11:36:46.876439 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:46.876400    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf","Type":"ContainerStarted","Data":"f16fd288d38b597aef507661b5671d408bc3da7e8feadffd9c2b41a66987a065"}
Sep 11 11:36:46.876844 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:46.876507    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf","Type":"ContainerStarted","Data":"cbb02c8bcc9041b9809742b10c85b3c13ef77269d21c3d3a5e60b77127dbe7e0"}
Sep 11 11:36:46.876844 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:46.876553    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf","Type":"ContainerStarted","Data":"2b85b69c60264c79794cfad16068bb7d08799a6e06faa84b9dc0b42daa7a53f5"}
Sep 11 11:36:46.876844 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:46.876588    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf","Type":"ContainerStarted","Data":"ef08080ded7438227bf663405d286fa4d62f3391a6521d886bdb0f86d1db5ece"}
Sep 11 11:36:46.876844 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:46.876624    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf","Type":"ContainerStarted","Data":"23237125c0d6ee88f7b96d447a614efc9329d17f46a0c1c69985d0dc8e053225"}
Sep 11 11:36:46.876844 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:46.876686    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf","Type":"ContainerStarted","Data":"513b8cb09933e3b5a9d25948e914364828b61b2b10e873d76e534ebda0d6a832"}
Sep 11 11:36:46.908017 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:46.907971    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.89327361 podStartE2EDuration="2.907954212s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.680602ms" isStatefulPod=false podCreationTimestamp="2026-09-11 11:36:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:36:46.906709324 +0000 UTC m=+147.327315353" watchObservedRunningTime="2026-09-11 11:36:46.907954212 +0000 UTC m=+147.328560238"
Sep 11 11:36:47.168685 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.168575    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:36:47.168930 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.168893    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="prometheus" containerID="cri-o://172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf" gracePeriod=600
Sep 11 11:36:47.169079 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.168942    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="thanos-sidecar" containerID="cri-o://e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910" gracePeriod=600
Sep 11 11:36:47.169079 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.168958    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="config-reloader" containerID="cri-o://a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872" gracePeriod=600
Sep 11 11:36:47.169079 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.168952    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="kube-rbac-proxy-thanos" containerID="cri-o://56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731" gracePeriod=600
Sep 11 11:36:47.169079 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.168987    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="kube-rbac-proxy-web" containerID="cri-o://0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d" gracePeriod=600
Sep 11 11:36:47.169079 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.168933    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="kube-rbac-proxy" containerID="cri-o://ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0" gracePeriod=600
Sep 11 11:36:47.521190 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.521164    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:47.610157 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.610124    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-thanos-prometheus-http-client-file\") pod \"44d30cae-3e69-4694-b8ab-5dd645d4e676\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") "
Sep 11 11:36:47.610334 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.610166    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rvrw\" (UniqueName: \"kubernetes.io/projected/44d30cae-3e69-4694-b8ab-5dd645d4e676-kube-api-access-8rvrw\") pod \"44d30cae-3e69-4694-b8ab-5dd645d4e676\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") "
Sep 11 11:36:47.610334 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.610190    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-web-config\") pod \"44d30cae-3e69-4694-b8ab-5dd645d4e676\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") "
Sep 11 11:36:47.610334 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.610220    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"44d30cae-3e69-4694-b8ab-5dd645d4e676\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") "
Sep 11 11:36:47.610498 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.610327    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-trusted-ca-bundle\") pod \"44d30cae-3e69-4694-b8ab-5dd645d4e676\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") "
Sep 11 11:36:47.610498 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.610392    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44d30cae-3e69-4694-b8ab-5dd645d4e676-tls-assets\") pod \"44d30cae-3e69-4694-b8ab-5dd645d4e676\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") "
Sep 11 11:36:47.610498 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.610420    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-grpc-tls\") pod \"44d30cae-3e69-4694-b8ab-5dd645d4e676\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") "
Sep 11 11:36:47.610498 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.610458    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-metrics-client-certs\") pod \"44d30cae-3e69-4694-b8ab-5dd645d4e676\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") "
Sep 11 11:36:47.610746 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.610536    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"44d30cae-3e69-4694-b8ab-5dd645d4e676\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") "
Sep 11 11:36:47.610746 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.610571    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-rulefiles-1\") pod \"44d30cae-3e69-4694-b8ab-5dd645d4e676\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") "
Sep 11 11:36:47.610746 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.610599    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-rulefiles-2\") pod \"44d30cae-3e69-4694-b8ab-5dd645d4e676\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") "
Sep 11 11:36:47.610746 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.610625    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-configmap-metrics-client-ca\") pod \"44d30cae-3e69-4694-b8ab-5dd645d4e676\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") "
Sep 11 11:36:47.610746 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.610684    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-prometheus-k8s-tls\") pod \"44d30cae-3e69-4694-b8ab-5dd645d4e676\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") "
Sep 11 11:36:47.610746 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.610714    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-db\") pod \"44d30cae-3e69-4694-b8ab-5dd645d4e676\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") "
Sep 11 11:36:47.610746 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.610747    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-configmap-serving-certs-ca-bundle\") pod \"44d30cae-3e69-4694-b8ab-5dd645d4e676\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") "
Sep 11 11:36:47.611249 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.610773    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-configmap-kubelet-serving-ca-bundle\") pod \"44d30cae-3e69-4694-b8ab-5dd645d4e676\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") "
Sep 11 11:36:47.611249 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.610800    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-rulefiles-0\") pod \"44d30cae-3e69-4694-b8ab-5dd645d4e676\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") "
Sep 11 11:36:47.611249 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.610822    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44d30cae-3e69-4694-b8ab-5dd645d4e676-config-out\") pod \"44d30cae-3e69-4694-b8ab-5dd645d4e676\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") "
Sep 11 11:36:47.611249 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.610845    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-config\") pod \"44d30cae-3e69-4694-b8ab-5dd645d4e676\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") "
Sep 11 11:36:47.611249 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.610876    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-kube-rbac-proxy\") pod \"44d30cae-3e69-4694-b8ab-5dd645d4e676\" (UID: \"44d30cae-3e69-4694-b8ab-5dd645d4e676\") "
Sep 11 11:36:47.611249 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.610977    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "44d30cae-3e69-4694-b8ab-5dd645d4e676" (UID: "44d30cae-3e69-4694-b8ab-5dd645d4e676"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:36:47.611249 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.611188    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-trusted-ca-bundle\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:47.612590 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.612334    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "44d30cae-3e69-4694-b8ab-5dd645d4e676" (UID: "44d30cae-3e69-4694-b8ab-5dd645d4e676"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:36:47.612992 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.612963    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "44d30cae-3e69-4694-b8ab-5dd645d4e676" (UID: "44d30cae-3e69-4694-b8ab-5dd645d4e676"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:36:47.613202 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.613176    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "44d30cae-3e69-4694-b8ab-5dd645d4e676" (UID: "44d30cae-3e69-4694-b8ab-5dd645d4e676"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:36:47.613816 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.613791    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "44d30cae-3e69-4694-b8ab-5dd645d4e676" (UID: "44d30cae-3e69-4694-b8ab-5dd645d4e676"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:36:47.614308 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.613907    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "44d30cae-3e69-4694-b8ab-5dd645d4e676" (UID: "44d30cae-3e69-4694-b8ab-5dd645d4e676"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:36:47.614529 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.614258    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "44d30cae-3e69-4694-b8ab-5dd645d4e676" (UID: "44d30cae-3e69-4694-b8ab-5dd645d4e676"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:36:47.614661 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.614353    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "44d30cae-3e69-4694-b8ab-5dd645d4e676" (UID: "44d30cae-3e69-4694-b8ab-5dd645d4e676"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:36:47.614661 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.614475    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "44d30cae-3e69-4694-b8ab-5dd645d4e676" (UID: "44d30cae-3e69-4694-b8ab-5dd645d4e676"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:36:47.614772 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.614662    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "44d30cae-3e69-4694-b8ab-5dd645d4e676" (UID: "44d30cae-3e69-4694-b8ab-5dd645d4e676"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:36:47.614922 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.614903    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44d30cae-3e69-4694-b8ab-5dd645d4e676-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "44d30cae-3e69-4694-b8ab-5dd645d4e676" (UID: "44d30cae-3e69-4694-b8ab-5dd645d4e676"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:36:47.615666 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.615618    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "44d30cae-3e69-4694-b8ab-5dd645d4e676" (UID: "44d30cae-3e69-4694-b8ab-5dd645d4e676"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:36:47.615955 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.615928    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44d30cae-3e69-4694-b8ab-5dd645d4e676-kube-api-access-8rvrw" (OuterVolumeSpecName: "kube-api-access-8rvrw") pod "44d30cae-3e69-4694-b8ab-5dd645d4e676" (UID: "44d30cae-3e69-4694-b8ab-5dd645d4e676"). InnerVolumeSpecName "kube-api-access-8rvrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:36:47.616078 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.616056    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "44d30cae-3e69-4694-b8ab-5dd645d4e676" (UID: "44d30cae-3e69-4694-b8ab-5dd645d4e676"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:36:47.616241 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.616215    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "44d30cae-3e69-4694-b8ab-5dd645d4e676" (UID: "44d30cae-3e69-4694-b8ab-5dd645d4e676"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:36:47.616305 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.616278    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "44d30cae-3e69-4694-b8ab-5dd645d4e676" (UID: "44d30cae-3e69-4694-b8ab-5dd645d4e676"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:36:47.616775 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.616751    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "44d30cae-3e69-4694-b8ab-5dd645d4e676" (UID: "44d30cae-3e69-4694-b8ab-5dd645d4e676"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:36:47.617305 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.617289    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-config" (OuterVolumeSpecName: "config") pod "44d30cae-3e69-4694-b8ab-5dd645d4e676" (UID: "44d30cae-3e69-4694-b8ab-5dd645d4e676"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:36:47.617418 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.617402    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44d30cae-3e69-4694-b8ab-5dd645d4e676-config-out" (OuterVolumeSpecName: "config-out") pod "44d30cae-3e69-4694-b8ab-5dd645d4e676" (UID: "44d30cae-3e69-4694-b8ab-5dd645d4e676"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:36:47.626830 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.626810    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-web-config" (OuterVolumeSpecName: "web-config") pod "44d30cae-3e69-4694-b8ab-5dd645d4e676" (UID: "44d30cae-3e69-4694-b8ab-5dd645d4e676"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:36:47.712413 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.712338    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-prometheus-k8s-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:47.712413 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.712363    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-db\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:47.712413 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.712373    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:47.712413 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.712384    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:47.712413 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.712393    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:47.712413 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.712401    2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/44d30cae-3e69-4694-b8ab-5dd645d4e676-config-out\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:47.712413 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.712411    2575 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-config\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:47.712413 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.712418    2575 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-kube-rbac-proxy\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:47.712780 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.712426    2575 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-thanos-prometheus-http-client-file\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:47.712780 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.712436    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rvrw\" (UniqueName: \"kubernetes.io/projected/44d30cae-3e69-4694-b8ab-5dd645d4e676-kube-api-access-8rvrw\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:47.712780 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.712444    2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-web-config\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:47.712780 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.712452    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:47.712780 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.712460    2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/44d30cae-3e69-4694-b8ab-5dd645d4e676-tls-assets\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:47.712780 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.712468    2575 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-grpc-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:47.712780 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.712475    2575 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-metrics-client-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:47.712780 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.712483    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44d30cae-3e69-4694-b8ab-5dd645d4e676-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:47.712780 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.712492    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:47.712780 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.712500    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:47.712780 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.712507    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44d30cae-3e69-4694-b8ab-5dd645d4e676-configmap-metrics-client-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:36:47.883845 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.883813    2575 generic.go:356] "Generic (PLEG): container finished" podID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerID="56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731" exitCode=0
Sep 11 11:36:47.883845 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.883836    2575 generic.go:356] "Generic (PLEG): container finished" podID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerID="ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0" exitCode=0
Sep 11 11:36:47.883845 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.883842    2575 generic.go:356] "Generic (PLEG): container finished" podID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerID="0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d" exitCode=0
Sep 11 11:36:47.883845 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.883847    2575 generic.go:356] "Generic (PLEG): container finished" podID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerID="e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910" exitCode=0
Sep 11 11:36:47.883845 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.883852    2575 generic.go:356] "Generic (PLEG): container finished" podID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerID="a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872" exitCode=0
Sep 11 11:36:47.884382 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.883857    2575 generic.go:356] "Generic (PLEG): container finished" podID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerID="172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf" exitCode=0
Sep 11 11:36:47.884382 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.883893    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:47.884382 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.883902    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44d30cae-3e69-4694-b8ab-5dd645d4e676","Type":"ContainerDied","Data":"56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731"}
Sep 11 11:36:47.884382 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.883969    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44d30cae-3e69-4694-b8ab-5dd645d4e676","Type":"ContainerDied","Data":"ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0"}
Sep 11 11:36:47.884382 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.883996    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44d30cae-3e69-4694-b8ab-5dd645d4e676","Type":"ContainerDied","Data":"0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d"}
Sep 11 11:36:47.884382 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.884022    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44d30cae-3e69-4694-b8ab-5dd645d4e676","Type":"ContainerDied","Data":"e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910"}
Sep 11 11:36:47.884382 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.884036    2575 scope.go:117] "RemoveContainer" containerID="56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731"
Sep 11 11:36:47.884382 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.884046    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44d30cae-3e69-4694-b8ab-5dd645d4e676","Type":"ContainerDied","Data":"a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872"}
Sep 11 11:36:47.884382 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.884070    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44d30cae-3e69-4694-b8ab-5dd645d4e676","Type":"ContainerDied","Data":"172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf"}
Sep 11 11:36:47.884382 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.884097    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"44d30cae-3e69-4694-b8ab-5dd645d4e676","Type":"ContainerDied","Data":"f5a3794d2d4cdbffccdfd6773b59b0e40bf1e3975993b9b6ce64fe2d119dda55"}
Sep 11 11:36:47.892625 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.892608    2575 scope.go:117] "RemoveContainer" containerID="ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0"
Sep 11 11:36:47.899916 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.899900    2575 scope.go:117] "RemoveContainer" containerID="0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d"
Sep 11 11:36:47.907119 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.907102    2575 scope.go:117] "RemoveContainer" containerID="e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910"
Sep 11 11:36:47.908219 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.908201    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:36:47.915114 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.915072    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:36:47.915180 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.915131    2575 scope.go:117] "RemoveContainer" containerID="a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872"
Sep 11 11:36:47.932385 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.932336    2575 scope.go:117] "RemoveContainer" containerID="172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf"
Sep 11 11:36:47.940682 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.940642    2575 scope.go:117] "RemoveContainer" containerID="35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4"
Sep 11 11:36:47.942359 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.942302    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:36:47.942652 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.942615    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="kube-rbac-proxy-thanos"
Sep 11 11:36:47.942652 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.942648    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="kube-rbac-proxy-thanos"
Sep 11 11:36:47.942776 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.942664    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="prometheus"
Sep 11 11:36:47.942776 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.942672    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="prometheus"
Sep 11 11:36:47.942776 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.942679    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="init-config-reloader"
Sep 11 11:36:47.942776 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.942685    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="init-config-reloader"
Sep 11 11:36:47.942776 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.942697    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="config-reloader"
Sep 11 11:36:47.942776 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.942702    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="config-reloader"
Sep 11 11:36:47.942776 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.942708    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="thanos-sidecar"
Sep 11 11:36:47.942776 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.942713    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="thanos-sidecar"
Sep 11 11:36:47.942776 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.942725    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="kube-rbac-proxy"
Sep 11 11:36:47.942776 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.942734    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="kube-rbac-proxy"
Sep 11 11:36:47.942776 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.942744    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="kube-rbac-proxy-web"
Sep 11 11:36:47.942776 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.942749    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="kube-rbac-proxy-web"
Sep 11 11:36:47.943127 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.942798    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="kube-rbac-proxy-web"
Sep 11 11:36:47.943127 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.942810    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="prometheus"
Sep 11 11:36:47.943127 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.942819    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="kube-rbac-proxy"
Sep 11 11:36:47.943127 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.942827    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="config-reloader"
Sep 11 11:36:47.943127 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.942833    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="thanos-sidecar"
Sep 11 11:36:47.943127 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.942838    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" containerName="kube-rbac-proxy-thanos"
Sep 11 11:36:47.948206 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.948185    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:47.951183 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.950311    2575 scope.go:117] "RemoveContainer" containerID="56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731"
Sep 11 11:36:47.951183 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:36:47.950689    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731\": container with ID starting with 56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731 not found: ID does not exist" containerID="56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731"
Sep 11 11:36:47.951183 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.950722    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731"} err="failed to get container status \"56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731\": rpc error: code = NotFound desc = could not find container \"56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731\": container with ID starting with 56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731 not found: ID does not exist"
Sep 11 11:36:47.951183 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.950747    2575 scope.go:117] "RemoveContainer" containerID="ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0"
Sep 11 11:36:47.951183 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:36:47.951146    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0\": container with ID starting with ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0 not found: ID does not exist" containerID="ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0"
Sep 11 11:36:47.951183 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.951173    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0"} err="failed to get container status \"ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0\": rpc error: code = NotFound desc = could not find container \"ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0\": container with ID starting with ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0 not found: ID does not exist"
Sep 11 11:36:47.951891 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.951191    2575 scope.go:117] "RemoveContainer" containerID="0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d"
Sep 11 11:36:47.951891 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.951335    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 11:36:47.951891 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.951591    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 11:36:47.952022 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.951924    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 11:36:47.952807 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.952284    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-jx9nk\""
Sep 11 11:36:47.952807 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.952449    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fv1qua16vuedl\""
Sep 11 11:36:47.952807 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.952596    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 11:36:47.952807 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.952606    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 11:36:47.953050 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.952832    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 11:36:47.953050 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.952859    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 11:36:47.953050 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.952889    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 11:36:47.953050 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.953012    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 11:36:47.953255 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.953053    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 11:36:47.953255 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.953060    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 11:36:47.953255 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.953147    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 11:36:47.953255 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:36:47.953173    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d\": container with ID starting with 0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d not found: ID does not exist" containerID="0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d"
Sep 11 11:36:47.953255 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.953207    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d"} err="failed to get container status \"0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d\": rpc error: code = NotFound desc = could not find container \"0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d\": container with ID starting with 0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d not found: ID does not exist"
Sep 11 11:36:47.953255 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.953226    2575 scope.go:117] "RemoveContainer" containerID="e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910"
Sep 11 11:36:47.953791 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:36:47.953663    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910\": container with ID starting with e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910 not found: ID does not exist" containerID="e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910"
Sep 11 11:36:47.953791 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.953694    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910"} err="failed to get container status \"e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910\": rpc error: code = NotFound desc = could not find container \"e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910\": container with ID starting with e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910 not found: ID does not exist"
Sep 11 11:36:47.953791 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.953714    2575 scope.go:117] "RemoveContainer" containerID="a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872"
Sep 11 11:36:47.954215 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:36:47.954179    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872\": container with ID starting with a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872 not found: ID does not exist" containerID="a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872"
Sep 11 11:36:47.954379 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.954351    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872"} err="failed to get container status \"a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872\": rpc error: code = NotFound desc = could not find container \"a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872\": container with ID starting with a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872 not found: ID does not exist"
Sep 11 11:36:47.954469 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.954458    2575 scope.go:117] "RemoveContainer" containerID="172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf"
Sep 11 11:36:47.955179 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:36:47.955082    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf\": container with ID starting with 172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf not found: ID does not exist" containerID="172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf"
Sep 11 11:36:47.955315 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.955195    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf"} err="failed to get container status \"172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf\": rpc error: code = NotFound desc = could not find container \"172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf\": container with ID starting with 172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf not found: ID does not exist"
Sep 11 11:36:47.955315 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.955217    2575 scope.go:117] "RemoveContainer" containerID="35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4"
Sep 11 11:36:47.955657 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:36:47.955611    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4\": container with ID starting with 35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4 not found: ID does not exist" containerID="35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4"
Sep 11 11:36:47.955783 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.955669    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4"} err="failed to get container status \"35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4\": rpc error: code = NotFound desc = could not find container \"35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4\": container with ID starting with 35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4 not found: ID does not exist"
Sep 11 11:36:47.955783 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.955691    2575 scope.go:117] "RemoveContainer" containerID="56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731"
Sep 11 11:36:47.956294 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.956239    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731"} err="failed to get container status \"56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731\": rpc error: code = NotFound desc = could not find container \"56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731\": container with ID starting with 56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731 not found: ID does not exist"
Sep 11 11:36:47.956382 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.956295    2575 scope.go:117] "RemoveContainer" containerID="ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0"
Sep 11 11:36:47.956655 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.956599    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0"} err="failed to get container status \"ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0\": rpc error: code = NotFound desc = could not find container \"ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0\": container with ID starting with ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0 not found: ID does not exist"
Sep 11 11:36:47.956724 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.956659    2575 scope.go:117] "RemoveContainer" containerID="0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d"
Sep 11 11:36:47.956952 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.956929    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d"} err="failed to get container status \"0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d\": rpc error: code = NotFound desc = could not find container \"0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d\": container with ID starting with 0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d not found: ID does not exist"
Sep 11 11:36:47.957006 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.956956    2575 scope.go:117] "RemoveContainer" containerID="e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910"
Sep 11 11:36:47.957283 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.957252    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910"} err="failed to get container status \"e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910\": rpc error: code = NotFound desc = could not find container \"e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910\": container with ID starting with e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910 not found: ID does not exist"
Sep 11 11:36:47.957373 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.957287    2575 scope.go:117] "RemoveContainer" containerID="a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872"
Sep 11 11:36:47.957611 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.957586    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872"} err="failed to get container status \"a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872\": rpc error: code = NotFound desc = could not find container \"a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872\": container with ID starting with a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872 not found: ID does not exist"
Sep 11 11:36:47.958010 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.957615    2575 scope.go:117] "RemoveContainer" containerID="172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf"
Sep 11 11:36:47.959297 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.959201    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf"} err="failed to get container status \"172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf\": rpc error: code = NotFound desc = could not find container \"172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf\": container with ID starting with 172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf not found: ID does not exist"
Sep 11 11:36:47.959297 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.959262    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 11:36:47.959459 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.959411    2575 scope.go:117] "RemoveContainer" containerID="35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4"
Sep 11 11:36:47.959748 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.959728    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4"} err="failed to get container status \"35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4\": rpc error: code = NotFound desc = could not find container \"35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4\": container with ID starting with 35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4 not found: ID does not exist"
Sep 11 11:36:47.959839 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.959749    2575 scope.go:117] "RemoveContainer" containerID="56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731"
Sep 11 11:36:47.960057 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.960028    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731"} err="failed to get container status \"56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731\": rpc error: code = NotFound desc = could not find container \"56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731\": container with ID starting with 56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731 not found: ID does not exist"
Sep 11 11:36:47.960057 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.960055    2575 scope.go:117] "RemoveContainer" containerID="ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0"
Sep 11 11:36:47.960244 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.960201    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 11:36:47.960404 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.960367    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0"} err="failed to get container status \"ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0\": rpc error: code = NotFound desc = could not find container \"ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0\": container with ID starting with ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0 not found: ID does not exist"
Sep 11 11:36:47.960495 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.960405    2575 scope.go:117] "RemoveContainer" containerID="0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d"
Sep 11 11:36:47.960876 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.960855    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d"} err="failed to get container status \"0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d\": rpc error: code = NotFound desc = could not find container \"0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d\": container with ID starting with 0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d not found: ID does not exist"
Sep 11 11:36:47.960876 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.960873    2575 scope.go:117] "RemoveContainer" containerID="e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910"
Sep 11 11:36:47.962121 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.961224    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910"} err="failed to get container status \"e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910\": rpc error: code = NotFound desc = could not find container \"e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910\": container with ID starting with e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910 not found: ID does not exist"
Sep 11 11:36:47.962121 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.961250    2575 scope.go:117] "RemoveContainer" containerID="a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872"
Sep 11 11:36:47.962395 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.962117    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872"} err="failed to get container status \"a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872\": rpc error: code = NotFound desc = could not find container \"a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872\": container with ID starting with a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872 not found: ID does not exist"
Sep 11 11:36:47.962395 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.962162    2575 scope.go:117] "RemoveContainer" containerID="172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf"
Sep 11 11:36:47.962543 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.962467    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf"} err="failed to get container status \"172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf\": rpc error: code = NotFound desc = could not find container \"172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf\": container with ID starting with 172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf not found: ID does not exist"
Sep 11 11:36:47.962543 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.962496    2575 scope.go:117] "RemoveContainer" containerID="35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4"
Sep 11 11:36:47.962804 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.962782    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4"} err="failed to get container status \"35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4\": rpc error: code = NotFound desc = could not find container \"35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4\": container with ID starting with 35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4 not found: ID does not exist"
Sep 11 11:36:47.962891 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.962805    2575 scope.go:117] "RemoveContainer" containerID="56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731"
Sep 11 11:36:47.963085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.963036    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731"} err="failed to get container status \"56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731\": rpc error: code = NotFound desc = could not find container \"56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731\": container with ID starting with 56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731 not found: ID does not exist"
Sep 11 11:36:47.963085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.963058    2575 scope.go:117] "RemoveContainer" containerID="ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0"
Sep 11 11:36:47.963269 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.963253    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0"} err="failed to get container status \"ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0\": rpc error: code = NotFound desc = could not find container \"ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0\": container with ID starting with ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0 not found: ID does not exist"
Sep 11 11:36:47.963347 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.963270    2575 scope.go:117] "RemoveContainer" containerID="0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d"
Sep 11 11:36:47.963521 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.963504    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d"} err="failed to get container status \"0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d\": rpc error: code = NotFound desc = could not find container \"0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d\": container with ID starting with 0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d not found: ID does not exist"
Sep 11 11:36:47.963591 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.963522    2575 scope.go:117] "RemoveContainer" containerID="e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910"
Sep 11 11:36:47.963781 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.963752    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910"} err="failed to get container status \"e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910\": rpc error: code = NotFound desc = could not find container \"e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910\": container with ID starting with e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910 not found: ID does not exist"
Sep 11 11:36:47.963781 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.963779    2575 scope.go:117] "RemoveContainer" containerID="a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872"
Sep 11 11:36:47.963898 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.963810    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:36:47.964068 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.964047    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872"} err="failed to get container status \"a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872\": rpc error: code = NotFound desc = could not find container \"a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872\": container with ID starting with a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872 not found: ID does not exist"
Sep 11 11:36:47.964113 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.964071    2575 scope.go:117] "RemoveContainer" containerID="172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf"
Sep 11 11:36:47.964328 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.964307    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf"} err="failed to get container status \"172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf\": rpc error: code = NotFound desc = could not find container \"172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf\": container with ID starting with 172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf not found: ID does not exist"
Sep 11 11:36:47.964379 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.964329    2575 scope.go:117] "RemoveContainer" containerID="35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4"
Sep 11 11:36:47.964624 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.964605    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4"} err="failed to get container status \"35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4\": rpc error: code = NotFound desc = could not find container \"35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4\": container with ID starting with 35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4 not found: ID does not exist"
Sep 11 11:36:47.964704 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.964625    2575 scope.go:117] "RemoveContainer" containerID="56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731"
Sep 11 11:36:47.964898 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.964873    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731"} err="failed to get container status \"56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731\": rpc error: code = NotFound desc = could not find container \"56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731\": container with ID starting with 56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731 not found: ID does not exist"
Sep 11 11:36:47.964898 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.964899    2575 scope.go:117] "RemoveContainer" containerID="ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0"
Sep 11 11:36:47.965534 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.965509    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0"} err="failed to get container status \"ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0\": rpc error: code = NotFound desc = could not find container \"ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0\": container with ID starting with ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0 not found: ID does not exist"
Sep 11 11:36:47.965534 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.965534    2575 scope.go:117] "RemoveContainer" containerID="0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d"
Sep 11 11:36:47.965840 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.965818    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d"} err="failed to get container status \"0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d\": rpc error: code = NotFound desc = could not find container \"0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d\": container with ID starting with 0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d not found: ID does not exist"
Sep 11 11:36:47.965923 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.965844    2575 scope.go:117] "RemoveContainer" containerID="e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910"
Sep 11 11:36:47.966141 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.966119    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910"} err="failed to get container status \"e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910\": rpc error: code = NotFound desc = could not find container \"e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910\": container with ID starting with e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910 not found: ID does not exist"
Sep 11 11:36:47.966197 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.966144    2575 scope.go:117] "RemoveContainer" containerID="a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872"
Sep 11 11:36:47.966469 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.966449    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872"} err="failed to get container status \"a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872\": rpc error: code = NotFound desc = could not find container \"a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872\": container with ID starting with a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872 not found: ID does not exist"
Sep 11 11:36:47.966522 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.966471    2575 scope.go:117] "RemoveContainer" containerID="172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf"
Sep 11 11:36:47.966812 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.966792    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf"} err="failed to get container status \"172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf\": rpc error: code = NotFound desc = could not find container \"172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf\": container with ID starting with 172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf not found: ID does not exist"
Sep 11 11:36:47.966812 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.966811    2575 scope.go:117] "RemoveContainer" containerID="35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4"
Sep 11 11:36:47.967099 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.967077    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4"} err="failed to get container status \"35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4\": rpc error: code = NotFound desc = could not find container \"35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4\": container with ID starting with 35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4 not found: ID does not exist"
Sep 11 11:36:47.967166 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.967103    2575 scope.go:117] "RemoveContainer" containerID="56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731"
Sep 11 11:36:47.967416 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.967394    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731"} err="failed to get container status \"56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731\": rpc error: code = NotFound desc = could not find container \"56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731\": container with ID starting with 56c0ffe24cefbff1df6f757ce2c1a73a5210fc226170a5a1566496b1abb45731 not found: ID does not exist"
Sep 11 11:36:47.967473 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.967420    2575 scope.go:117] "RemoveContainer" containerID="ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0"
Sep 11 11:36:47.967721 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.967700    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0"} err="failed to get container status \"ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0\": rpc error: code = NotFound desc = could not find container \"ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0\": container with ID starting with ce2643b2c1b9d39920938b51a4b3501fa451b719909bed8c77fbe7b5ef1d31d0 not found: ID does not exist"
Sep 11 11:36:47.967721 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.967721    2575 scope.go:117] "RemoveContainer" containerID="0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d"
Sep 11 11:36:47.967970 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.967952    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d"} err="failed to get container status \"0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d\": rpc error: code = NotFound desc = could not find container \"0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d\": container with ID starting with 0d12787d3af3ff34804fd858a0fc8cf122cdf98d991dfac154722b7456f8ef0d not found: ID does not exist"
Sep 11 11:36:47.967970 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.967970    2575 scope.go:117] "RemoveContainer" containerID="e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910"
Sep 11 11:36:47.968212 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.968193    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910"} err="failed to get container status \"e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910\": rpc error: code = NotFound desc = could not find container \"e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910\": container with ID starting with e853f77f945c39e99b525421165f523980057754b1d8b5d78dc00688d48cd910 not found: ID does not exist"
Sep 11 11:36:47.968212 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.968212    2575 scope.go:117] "RemoveContainer" containerID="a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872"
Sep 11 11:36:47.968451 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.968433    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872"} err="failed to get container status \"a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872\": rpc error: code = NotFound desc = could not find container \"a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872\": container with ID starting with a2b906bf3b185e1f343e5b1c281fd24f3235d62c20c287d107136459a8e8b872 not found: ID does not exist"
Sep 11 11:36:47.968451 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.968449    2575 scope.go:117] "RemoveContainer" containerID="172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf"
Sep 11 11:36:47.968685 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.968666    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf"} err="failed to get container status \"172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf\": rpc error: code = NotFound desc = could not find container \"172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf\": container with ID starting with 172e6e433b2029bf083c2c919f86b186c84b296991fc46977abf10f57d70e3bf not found: ID does not exist"
Sep 11 11:36:47.968685 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.968684    2575 scope.go:117] "RemoveContainer" containerID="35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4"
Sep 11 11:36:47.968894 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:47.968878    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4"} err="failed to get container status \"35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4\": rpc error: code = NotFound desc = could not find container \"35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4\": container with ID starting with 35a37447f7cec6a08763c3beaacbd1cb31f24d6e45e430af2fa76d68cdc538f4 not found: ID does not exist"
Sep 11 11:36:48.014885 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.014856    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.014885 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.014887    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.015067 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.014918    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.015067 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.014934    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j9f5\" (UniqueName: \"kubernetes.io/projected/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-kube-api-access-9j9f5\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.015067 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.014991    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.015067 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.015019    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.015067 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.015039    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.015300 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.015086    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.015300 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.015119    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.015300 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.015146    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-web-config\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.015300 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.015177    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-config-out\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.015300 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.015208    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.015300 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.015240    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.015300 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.015275    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.015568 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.015329    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-config\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.015568 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.015343    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.015568 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.015361    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.015568 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.015386    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.015568 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.015427    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.015568 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.015442    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.081400 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.081368    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44d30cae-3e69-4694-b8ab-5dd645d4e676" path="/var/lib/kubelet/pods/44d30cae-3e69-4694-b8ab-5dd645d4e676/volumes"
Sep 11 11:36:48.115935 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.115912    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.115989 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.115940    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j9f5\" (UniqueName: \"kubernetes.io/projected/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-kube-api-access-9j9f5\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.115989 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.115959    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.115989 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.115979    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.116142 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.115995    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.116142 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.116012    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.116142 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.116030    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.116142 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.116052    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-web-config\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.116142 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.116078    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-config-out\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.116142 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.116104    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.116142 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.116126    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.116453 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.116152    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.116453 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.116205    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-config\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.116453 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.116225    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.116453 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.116248    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.116453 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.116282    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.116453 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.116323    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.116453 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.116348    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.116453 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.116397    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.116453 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.116421    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.117062 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.116694    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.117062 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.116805    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.117062 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.117006    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.117296 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.117276    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.117682 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.117661    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.118375 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.118351    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.119571 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.119251    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.119853 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.119831    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-config-out\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.119929 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.119877    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-config\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.120268 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.120244    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.120343 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.120308    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-web-config\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.120709 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.120684    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.121594 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.120985    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.121594 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.121554    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.121986 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.121959    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.122543 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.122517    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.122667 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.122602    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.122857 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.122836    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.123485 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.123462    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.125008 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.124989    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j9f5\" (UniqueName: \"kubernetes.io/projected/6172b2cd-5c4a-4ecf-9aff-2cb30284ef01-kube-api-access-9j9f5\") pod \"prometheus-k8s-0\" (UID: \"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.262717 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.262690    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:48.544735 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.544711    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 11:36:48.547138 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:36:48.547111    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6172b2cd_5c4a_4ecf_9aff_2cb30284ef01.slice/crio-4b6c1be1337fdfcd5ff5d41abe4873f7ce085d64a276eb3495cdf38e5e85254e WatchSource:0}: Error finding container 4b6c1be1337fdfcd5ff5d41abe4873f7ce085d64a276eb3495cdf38e5e85254e: Status 404 returned error can't find the container with id 4b6c1be1337fdfcd5ff5d41abe4873f7ce085d64a276eb3495cdf38e5e85254e
Sep 11 11:36:48.891426 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.891337    2575 generic.go:356] "Generic (PLEG): container finished" podID="6172b2cd-5c4a-4ecf-9aff-2cb30284ef01" containerID="aa5122149e8f24016c62fa4f95771ef136a2dd48c8ecb2a7423cf81cb83ce778" exitCode=0
Sep 11 11:36:48.891426 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.891379    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01","Type":"ContainerDied","Data":"aa5122149e8f24016c62fa4f95771ef136a2dd48c8ecb2a7423cf81cb83ce778"}
Sep 11 11:36:48.891426 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:48.891425    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01","Type":"ContainerStarted","Data":"4b6c1be1337fdfcd5ff5d41abe4873f7ce085d64a276eb3495cdf38e5e85254e"}
Sep 11 11:36:49.903829 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:49.903797    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01","Type":"ContainerStarted","Data":"be1f0cff8104b7d00829b9b56e5c8f4a5799c119a9bdf416922d989adbd52cb6"}
Sep 11 11:36:49.904289 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:49.903898    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01","Type":"ContainerStarted","Data":"6dd54de59d607915c379014aeca52272ed93c77c8cdc005c395c9982df1d0eb9"}
Sep 11 11:36:49.904289 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:49.903945    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01","Type":"ContainerStarted","Data":"3148b7dbc73c284437b5da502d303453476188f8cc0bb22bf95e6ba82d127a48"}
Sep 11 11:36:49.904289 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:49.903987    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01","Type":"ContainerStarted","Data":"d7a38d796fd00a0e07addc1d60370a83ab5130528dce473bbf8ed0477f6265ba"}
Sep 11 11:36:49.904289 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:49.904023    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01","Type":"ContainerStarted","Data":"f470b613286779c02eb28fd1113946c352359b106ad3920b4fc04243029832a4"}
Sep 11 11:36:49.904289 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:49.904070    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6172b2cd-5c4a-4ecf-9aff-2cb30284ef01","Type":"ContainerStarted","Data":"4d3422d8afe774b93c1d38cb10f173a0ed6ed916d0fbf8d8db167231f9f55f0a"}
Sep 11 11:36:49.934157 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:49.934111    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.915280341 podStartE2EDuration="2.934093046s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.812705ms" isStatefulPod=false podCreationTimestamp="2026-09-11 11:36:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:36:49.931399247 +0000 UTC m=+150.352005278" watchObservedRunningTime="2026-09-11 11:36:49.934093046 +0000 UTC m=+150.354699070"
Sep 11 11:36:50.654889 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:50.654852    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:36:50.709565 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:50.709532    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:36:53.263462 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:53.263432    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:36:53.388118 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:53.388090    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:53.388287 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:53.388144    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:53.392937 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:53.392918    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:53.717039 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:53.716946    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:36:53.924340 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:53.924310    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:36:53.973354 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:53.973281    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7df75766fd-lfdzx"]
Sep 11 11:36:54.091186 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:54.091139    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:36:54.159604 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:54.159573    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:36:54.286882 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:54.286845    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:36:54.390654 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:36:54.390596    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:37:04.019523 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:04.019487    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:37:04.231964 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:04.231920    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:37:04.301765 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:04.301680    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:37:05.497793 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:05.497756    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:37:18.993838 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:18.993729    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7df75766fd-lfdzx" podUID="6457a6e8-f976-416c-a169-4f51b07be349" containerName="console" containerID="cri-o://3e52c3436968468bc97cb5bd1de90c3b3dca70bc5d27f27133b30940d0bf745c" gracePeriod=15
Sep 11 11:37:19.245727 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:19.245667    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:37:19.395060 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:19.395022    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-trusted-ca-bundle\") pod \"6457a6e8-f976-416c-a169-4f51b07be349\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") "
Sep 11 11:37:19.395248 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:19.395066    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hshxf\" (UniqueName: \"kubernetes.io/projected/6457a6e8-f976-416c-a169-4f51b07be349-kube-api-access-hshxf\") pod \"6457a6e8-f976-416c-a169-4f51b07be349\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") "
Sep 11 11:37:19.395248 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:19.395095    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6457a6e8-f976-416c-a169-4f51b07be349-console-serving-cert\") pod \"6457a6e8-f976-416c-a169-4f51b07be349\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") "
Sep 11 11:37:19.395248 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:19.395164    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-console-config\") pod \"6457a6e8-f976-416c-a169-4f51b07be349\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") "
Sep 11 11:37:19.395248 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:19.395214    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-service-ca\") pod \"6457a6e8-f976-416c-a169-4f51b07be349\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") "
Sep 11 11:37:19.395248 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:19.395244    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-oauth-serving-cert\") pod \"6457a6e8-f976-416c-a169-4f51b07be349\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") "
Sep 11 11:37:19.395497 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:19.395288    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6457a6e8-f976-416c-a169-4f51b07be349-console-oauth-config\") pod \"6457a6e8-f976-416c-a169-4f51b07be349\" (UID: \"6457a6e8-f976-416c-a169-4f51b07be349\") "
Sep 11 11:37:19.395575 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:19.395444    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6457a6e8-f976-416c-a169-4f51b07be349" (UID: "6457a6e8-f976-416c-a169-4f51b07be349"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:37:19.395662 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:19.395595    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-console-config" (OuterVolumeSpecName: "console-config") pod "6457a6e8-f976-416c-a169-4f51b07be349" (UID: "6457a6e8-f976-416c-a169-4f51b07be349"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:37:19.395662 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:19.395606    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-service-ca" (OuterVolumeSpecName: "service-ca") pod "6457a6e8-f976-416c-a169-4f51b07be349" (UID: "6457a6e8-f976-416c-a169-4f51b07be349"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:37:19.395662 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:19.395623    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6457a6e8-f976-416c-a169-4f51b07be349" (UID: "6457a6e8-f976-416c-a169-4f51b07be349"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:37:19.397605 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:19.397583    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6457a6e8-f976-416c-a169-4f51b07be349-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6457a6e8-f976-416c-a169-4f51b07be349" (UID: "6457a6e8-f976-416c-a169-4f51b07be349"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:37:19.397908 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:19.397882    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6457a6e8-f976-416c-a169-4f51b07be349-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6457a6e8-f976-416c-a169-4f51b07be349" (UID: "6457a6e8-f976-416c-a169-4f51b07be349"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:37:19.398006 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:19.397912    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6457a6e8-f976-416c-a169-4f51b07be349-kube-api-access-hshxf" (OuterVolumeSpecName: "kube-api-access-hshxf") pod "6457a6e8-f976-416c-a169-4f51b07be349" (UID: "6457a6e8-f976-416c-a169-4f51b07be349"). InnerVolumeSpecName "kube-api-access-hshxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:37:19.495994 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:19.495909    2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-console-config\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:37:19.495994 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:19.495936    2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-service-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:37:19.495994 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:19.495945    2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-oauth-serving-cert\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:37:19.495994 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:19.495954    2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6457a6e8-f976-416c-a169-4f51b07be349-console-oauth-config\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:37:19.495994 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:19.495962    2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6457a6e8-f976-416c-a169-4f51b07be349-trusted-ca-bundle\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:37:19.495994 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:19.495970    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hshxf\" (UniqueName: \"kubernetes.io/projected/6457a6e8-f976-416c-a169-4f51b07be349-kube-api-access-hshxf\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:37:19.495994 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:19.495978    2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6457a6e8-f976-416c-a169-4f51b07be349-console-serving-cert\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:37:20.031685 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:20.031654    2575 generic.go:356] "Generic (PLEG): container finished" podID="6457a6e8-f976-416c-a169-4f51b07be349" containerID="3e52c3436968468bc97cb5bd1de90c3b3dca70bc5d27f27133b30940d0bf745c" exitCode=0
Sep 11 11:37:20.032092 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:20.031689    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7df75766fd-lfdzx"
Sep 11 11:37:20.032092 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:20.031726    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7df75766fd-lfdzx" event={"ID":"6457a6e8-f976-416c-a169-4f51b07be349","Type":"ContainerDied","Data":"3e52c3436968468bc97cb5bd1de90c3b3dca70bc5d27f27133b30940d0bf745c"}
Sep 11 11:37:20.032092 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:20.031776    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7df75766fd-lfdzx" event={"ID":"6457a6e8-f976-416c-a169-4f51b07be349","Type":"ContainerDied","Data":"40e4998e892f59532545a640acab1a992903029ece1b4df6c6ca022363f60889"}
Sep 11 11:37:20.032092 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:20.031816    2575 scope.go:117] "RemoveContainer" containerID="3e52c3436968468bc97cb5bd1de90c3b3dca70bc5d27f27133b30940d0bf745c"
Sep 11 11:37:20.040031 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:20.040017    2575 scope.go:117] "RemoveContainer" containerID="3e52c3436968468bc97cb5bd1de90c3b3dca70bc5d27f27133b30940d0bf745c"
Sep 11 11:37:20.040291 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:37:20.040273    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e52c3436968468bc97cb5bd1de90c3b3dca70bc5d27f27133b30940d0bf745c\": container with ID starting with 3e52c3436968468bc97cb5bd1de90c3b3dca70bc5d27f27133b30940d0bf745c not found: ID does not exist" containerID="3e52c3436968468bc97cb5bd1de90c3b3dca70bc5d27f27133b30940d0bf745c"
Sep 11 11:37:20.040346 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:20.040297    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e52c3436968468bc97cb5bd1de90c3b3dca70bc5d27f27133b30940d0bf745c"} err="failed to get container status \"3e52c3436968468bc97cb5bd1de90c3b3dca70bc5d27f27133b30940d0bf745c\": rpc error: code = NotFound desc = could not find container \"3e52c3436968468bc97cb5bd1de90c3b3dca70bc5d27f27133b30940d0bf745c\": container with ID starting with 3e52c3436968468bc97cb5bd1de90c3b3dca70bc5d27f27133b30940d0bf745c not found: ID does not exist"
Sep 11 11:37:20.051981 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:20.051950    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7df75766fd-lfdzx"]
Sep 11 11:37:20.053970 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:20.053945    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7df75766fd-lfdzx"]
Sep 11 11:37:20.083197 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:20.083170    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6457a6e8-f976-416c-a169-4f51b07be349" path="/var/lib/kubelet/pods/6457a6e8-f976-416c-a169-4f51b07be349/volumes"
Sep 11 11:37:33.512399 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:33.512358    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:37:33.565548 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:33.565519    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:37:36.248800 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:36.248773    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:37:36.346884 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:36.346846    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:37:48.263552 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:48.263519    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:37:48.279074 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:48.279034    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:37:49.157868 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:37:49.157843    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 11:38:24.414982 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:38:24.414940    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:38:43.115529 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:38:43.115413    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:38:43.216479 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:38:43.216449    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:00.195725 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:00.195683    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:01.671792 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:01.671762    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:04.143227 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:04.143184    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:05.475680 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:05.475648    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:05.587842 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:05.587803    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:05.919958 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:05.919929    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:06.064873 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:06.064842    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:07.486144 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:07.486109    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:07.875149 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:07.875081    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:10.683327 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:10.683290    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:10.746096 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:10.746056    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:16.683647 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:16.683595    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:17.295764 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:17.295734    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xlgnr"]
Sep 11 11:39:17.296142 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:17.296125    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6457a6e8-f976-416c-a169-4f51b07be349" containerName="console"
Sep 11 11:39:17.296185 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:17.296144    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6457a6e8-f976-416c-a169-4f51b07be349" containerName="console"
Sep 11 11:39:17.296219 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:17.296199    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6457a6e8-f976-416c-a169-4f51b07be349" containerName="console"
Sep 11 11:39:17.298869 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:17.298850    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-xlgnr"
Sep 11 11:39:17.303688 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:17.303668    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 11 11:39:17.304527 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:17.304504    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-rppkq\""
Sep 11 11:39:17.304608 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:17.304511    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 11 11:39:17.315455 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:17.315435    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xlgnr"]
Sep 11 11:39:17.388218 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:17.388164    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt2gl\" (UniqueName: \"kubernetes.io/projected/1577c12a-3c4c-475f-b847-aaafb7043757-kube-api-access-nt2gl\") pod \"cert-manager-webhook-6486ffd997-xlgnr\" (UID: \"1577c12a-3c4c-475f-b847-aaafb7043757\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xlgnr"
Sep 11 11:39:17.388379 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:17.388279    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1577c12a-3c4c-475f-b847-aaafb7043757-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xlgnr\" (UID: \"1577c12a-3c4c-475f-b847-aaafb7043757\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xlgnr"
Sep 11 11:39:17.388841 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:17.388819    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:17.481014 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:17.480988    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:17.489499 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:17.489471    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt2gl\" (UniqueName: \"kubernetes.io/projected/1577c12a-3c4c-475f-b847-aaafb7043757-kube-api-access-nt2gl\") pod \"cert-manager-webhook-6486ffd997-xlgnr\" (UID: \"1577c12a-3c4c-475f-b847-aaafb7043757\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xlgnr"
Sep 11 11:39:17.489613 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:17.489544    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1577c12a-3c4c-475f-b847-aaafb7043757-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xlgnr\" (UID: \"1577c12a-3c4c-475f-b847-aaafb7043757\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xlgnr"
Sep 11 11:39:17.499347 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:17.499324    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1577c12a-3c4c-475f-b847-aaafb7043757-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xlgnr\" (UID: \"1577c12a-3c4c-475f-b847-aaafb7043757\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xlgnr"
Sep 11 11:39:17.499428 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:17.499395    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt2gl\" (UniqueName: \"kubernetes.io/projected/1577c12a-3c4c-475f-b847-aaafb7043757-kube-api-access-nt2gl\") pod \"cert-manager-webhook-6486ffd997-xlgnr\" (UID: \"1577c12a-3c4c-475f-b847-aaafb7043757\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xlgnr"
Sep 11 11:39:17.628960 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:17.628897    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-xlgnr"
Sep 11 11:39:17.896815 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:17.896739    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xlgnr"]
Sep 11 11:39:17.900366 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:39:17.900339    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1577c12a_3c4c_475f_b847_aaafb7043757.slice/crio-5d77d08e71a4951dfec8effabc62ba3b61eff75565c9f39dad09b8ff96511a2f WatchSource:0}: Error finding container 5d77d08e71a4951dfec8effabc62ba3b61eff75565c9f39dad09b8ff96511a2f: Status 404 returned error can't find the container with id 5d77d08e71a4951dfec8effabc62ba3b61eff75565c9f39dad09b8ff96511a2f
Sep 11 11:39:18.496137 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:18.496109    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-xlgnr" event={"ID":"1577c12a-3c4c-475f-b847-aaafb7043757","Type":"ContainerStarted","Data":"5d77d08e71a4951dfec8effabc62ba3b61eff75565c9f39dad09b8ff96511a2f"}
Sep 11 11:39:19.313810 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:19.313768    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mjz84"]
Sep 11 11:39:19.317472 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:19.317446    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mjz84"
Sep 11 11:39:19.320039 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:19.320019    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 11:39:19.321399 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:19.321158    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 11:39:19.321399 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:19.321217    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-bh4ms\""
Sep 11 11:39:19.324973 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:19.324927    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mjz84"]
Sep 11 11:39:19.408386 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:19.408299    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/889c079f-f425-40c9-9669-e141d2b60c46-tmp\") pod \"openshift-lws-operator-fd8ccff4c-mjz84\" (UID: \"889c079f-f425-40c9-9669-e141d2b60c46\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mjz84"
Sep 11 11:39:19.408539 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:19.408469    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwpcg\" (UniqueName: \"kubernetes.io/projected/889c079f-f425-40c9-9669-e141d2b60c46-kube-api-access-jwpcg\") pod \"openshift-lws-operator-fd8ccff4c-mjz84\" (UID: \"889c079f-f425-40c9-9669-e141d2b60c46\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mjz84"
Sep 11 11:39:19.509874 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:19.509840    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/889c079f-f425-40c9-9669-e141d2b60c46-tmp\") pod \"openshift-lws-operator-fd8ccff4c-mjz84\" (UID: \"889c079f-f425-40c9-9669-e141d2b60c46\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mjz84"
Sep 11 11:39:19.510061 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:19.509941    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwpcg\" (UniqueName: \"kubernetes.io/projected/889c079f-f425-40c9-9669-e141d2b60c46-kube-api-access-jwpcg\") pod \"openshift-lws-operator-fd8ccff4c-mjz84\" (UID: \"889c079f-f425-40c9-9669-e141d2b60c46\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mjz84"
Sep 11 11:39:19.510271 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:19.510249    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/889c079f-f425-40c9-9669-e141d2b60c46-tmp\") pod \"openshift-lws-operator-fd8ccff4c-mjz84\" (UID: \"889c079f-f425-40c9-9669-e141d2b60c46\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mjz84"
Sep 11 11:39:19.519328 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:19.519304    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwpcg\" (UniqueName: \"kubernetes.io/projected/889c079f-f425-40c9-9669-e141d2b60c46-kube-api-access-jwpcg\") pod \"openshift-lws-operator-fd8ccff4c-mjz84\" (UID: \"889c079f-f425-40c9-9669-e141d2b60c46\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mjz84"
Sep 11 11:39:19.630847 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:19.630813    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mjz84"
Sep 11 11:39:21.020092 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:21.020063    2575 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 11:39:21.258115 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:21.258091    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mjz84"]
Sep 11 11:39:21.260810 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:39:21.260783    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod889c079f_f425_40c9_9669_e141d2b60c46.slice/crio-61b0aac535f96171afbfa8ba4677609fb814fecadbbdd18d9b87ec07689201d7 WatchSource:0}: Error finding container 61b0aac535f96171afbfa8ba4677609fb814fecadbbdd18d9b87ec07689201d7: Status 404 returned error can't find the container with id 61b0aac535f96171afbfa8ba4677609fb814fecadbbdd18d9b87ec07689201d7
Sep 11 11:39:21.262725 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:21.262703    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 11:39:21.306036 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:21.306014    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:21.374564 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:21.374535    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:21.512593 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:21.512524    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mjz84" event={"ID":"889c079f-f425-40c9-9669-e141d2b60c46","Type":"ContainerStarted","Data":"61b0aac535f96171afbfa8ba4677609fb814fecadbbdd18d9b87ec07689201d7"}
Sep 11 11:39:21.516357 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:21.516332    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-xlgnr" event={"ID":"1577c12a-3c4c-475f-b847-aaafb7043757","Type":"ContainerStarted","Data":"5e9438feedad177d0e41257ae18b63bfb94b07fa6028b9948142034619624811"}
Sep 11 11:39:21.516476 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:21.516386    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-xlgnr"
Sep 11 11:39:21.534978 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:21.534938    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-xlgnr" podStartSLOduration=1.382079777 podStartE2EDuration="4.534924326s" totalImagesPullingTime="3.152844549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:39:21.534068828 +0000 UTC m=+301.954674860" watchObservedRunningTime="2026-09-11 11:39:21.534924326 +0000 UTC m=+301.955530353"
Sep 11 11:39:24.532534 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:24.532500    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mjz84" event={"ID":"889c079f-f425-40c9-9669-e141d2b60c46","Type":"ContainerStarted","Data":"3fa89c5dcf9117ce30eac8eb151dc61162d9453829f0968c352a41d65162addf"}
Sep 11 11:39:24.550027 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:24.549985    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mjz84" podStartSLOduration=3.288775467 podStartE2EDuration="5.549970912s" totalImagesPullingTime="2.261195445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:39:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:39:24.548906719 +0000 UTC m=+304.969512745" watchObservedRunningTime="2026-09-11 11:39:24.549970912 +0000 UTC m=+304.970576939"
Sep 11 11:39:25.277302 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:25.277266    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:25.399181 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:25.399153    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:26.928226 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:26.928187    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:27.523498 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:27.523464    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-xlgnr"
Sep 11 11:39:39.147653 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:39.147605    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:39.259291 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:39.259258    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:39.526268 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:39.526233    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:39.885747 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:39.885656    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:40.000437 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.000385    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:40.025193 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.025160    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb"]
Sep 11 11:39:40.099419 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.099390    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb"
Sep 11 11:39:40.103152 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.103124    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 11:39:40.103356 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.103336    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jxm97\""
Sep 11 11:39:40.103500 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.103273    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 11:39:40.103976 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.103957    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 11:39:40.104202 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.104185    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 11:39:40.105778 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.105761    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb"]
Sep 11 11:39:40.190570 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.190506    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/579453ba-899a-4c77-9057-1b54a63865e3-webhook-cert\") pod \"opendatahub-operator-controller-manager-86f769d976-2s4jb\" (UID: \"579453ba-899a-4c77-9057-1b54a63865e3\") " pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb"
Sep 11 11:39:40.190570 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.190549    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/579453ba-899a-4c77-9057-1b54a63865e3-manifests\") pod \"opendatahub-operator-controller-manager-86f769d976-2s4jb\" (UID: \"579453ba-899a-4c77-9057-1b54a63865e3\") " pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb"
Sep 11 11:39:40.190929 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.190574    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/579453ba-899a-4c77-9057-1b54a63865e3-tmp\") pod \"opendatahub-operator-controller-manager-86f769d976-2s4jb\" (UID: \"579453ba-899a-4c77-9057-1b54a63865e3\") " pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb"
Sep 11 11:39:40.190929 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.190675    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/579453ba-899a-4c77-9057-1b54a63865e3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86f769d976-2s4jb\" (UID: \"579453ba-899a-4c77-9057-1b54a63865e3\") " pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb"
Sep 11 11:39:40.190929 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.190729    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xw9s\" (UniqueName: \"kubernetes.io/projected/579453ba-899a-4c77-9057-1b54a63865e3-kube-api-access-8xw9s\") pod \"opendatahub-operator-controller-manager-86f769d976-2s4jb\" (UID: \"579453ba-899a-4c77-9057-1b54a63865e3\") " pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb"
Sep 11 11:39:40.248344 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.248319    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:40.291986 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.291955    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/579453ba-899a-4c77-9057-1b54a63865e3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86f769d976-2s4jb\" (UID: \"579453ba-899a-4c77-9057-1b54a63865e3\") " pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb"
Sep 11 11:39:40.292114 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.292011    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xw9s\" (UniqueName: \"kubernetes.io/projected/579453ba-899a-4c77-9057-1b54a63865e3-kube-api-access-8xw9s\") pod \"opendatahub-operator-controller-manager-86f769d976-2s4jb\" (UID: \"579453ba-899a-4c77-9057-1b54a63865e3\") " pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb"
Sep 11 11:39:40.292114 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.292045    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/579453ba-899a-4c77-9057-1b54a63865e3-webhook-cert\") pod \"opendatahub-operator-controller-manager-86f769d976-2s4jb\" (UID: \"579453ba-899a-4c77-9057-1b54a63865e3\") " pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb"
Sep 11 11:39:40.292114 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.292078    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/579453ba-899a-4c77-9057-1b54a63865e3-manifests\") pod \"opendatahub-operator-controller-manager-86f769d976-2s4jb\" (UID: \"579453ba-899a-4c77-9057-1b54a63865e3\") " pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb"
Sep 11 11:39:40.292303 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.292116    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/579453ba-899a-4c77-9057-1b54a63865e3-tmp\") pod \"opendatahub-operator-controller-manager-86f769d976-2s4jb\" (UID: \"579453ba-899a-4c77-9057-1b54a63865e3\") " pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb"
Sep 11 11:39:40.292495 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.292469    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/579453ba-899a-4c77-9057-1b54a63865e3-manifests\") pod \"opendatahub-operator-controller-manager-86f769d976-2s4jb\" (UID: \"579453ba-899a-4c77-9057-1b54a63865e3\") " pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb"
Sep 11 11:39:40.294460 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.294437    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/579453ba-899a-4c77-9057-1b54a63865e3-tmp\") pod \"opendatahub-operator-controller-manager-86f769d976-2s4jb\" (UID: \"579453ba-899a-4c77-9057-1b54a63865e3\") " pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb"
Sep 11 11:39:40.294572 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.294538    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/579453ba-899a-4c77-9057-1b54a63865e3-webhook-cert\") pod \"opendatahub-operator-controller-manager-86f769d976-2s4jb\" (UID: \"579453ba-899a-4c77-9057-1b54a63865e3\") " pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb"
Sep 11 11:39:40.294692 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.294676    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/579453ba-899a-4c77-9057-1b54a63865e3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86f769d976-2s4jb\" (UID: \"579453ba-899a-4c77-9057-1b54a63865e3\") " pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb"
Sep 11 11:39:40.301085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.301065    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xw9s\" (UniqueName: \"kubernetes.io/projected/579453ba-899a-4c77-9057-1b54a63865e3-kube-api-access-8xw9s\") pod \"opendatahub-operator-controller-manager-86f769d976-2s4jb\" (UID: \"579453ba-899a-4c77-9057-1b54a63865e3\") " pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb"
Sep 11 11:39:40.369317 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.369255    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:40.411275 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.411248    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb"
Sep 11 11:39:40.718188 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:40.718164    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb"]
Sep 11 11:39:40.720718 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:39:40.720694    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod579453ba_899a_4c77_9057_1b54a63865e3.slice/crio-4a252d685689492a25dd1a6cf12bb41e1c9bf2b060d2901d4dbbc767547e17ad WatchSource:0}: Error finding container 4a252d685689492a25dd1a6cf12bb41e1c9bf2b060d2901d4dbbc767547e17ad: Status 404 returned error can't find the container with id 4a252d685689492a25dd1a6cf12bb41e1c9bf2b060d2901d4dbbc767547e17ad
Sep 11 11:39:41.605394 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:41.605310    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb" event={"ID":"579453ba-899a-4c77-9057-1b54a63865e3","Type":"ContainerStarted","Data":"4a252d685689492a25dd1a6cf12bb41e1c9bf2b060d2901d4dbbc767547e17ad"}
Sep 11 11:39:41.662426 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:41.662395    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:41.760447 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:41.760411    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:43.616354 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:43.616322    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb" event={"ID":"579453ba-899a-4c77-9057-1b54a63865e3","Type":"ContainerStarted","Data":"4ed6a9d519fb106a94a6313192c4133ba91d464cf8cbddcfb6db861b61e13540"}
Sep 11 11:39:45.626076 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:45.626041    2575 generic.go:356] "Generic (PLEG): container finished" podID="579453ba-899a-4c77-9057-1b54a63865e3" containerID="4ed6a9d519fb106a94a6313192c4133ba91d464cf8cbddcfb6db861b61e13540" exitCode=0
Sep 11 11:39:45.626445 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:45.626117    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb" event={"ID":"579453ba-899a-4c77-9057-1b54a63865e3","Type":"ContainerDied","Data":"4ed6a9d519fb106a94a6313192c4133ba91d464cf8cbddcfb6db861b61e13540"}
Sep 11 11:39:46.632931 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:46.632889    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb" event={"ID":"579453ba-899a-4c77-9057-1b54a63865e3","Type":"ContainerStarted","Data":"eadad3558ce0f6dbe7d1aaaf90aecf718266a1f377d2dbdd2f7b54233859b8b0"}
Sep 11 11:39:46.633361 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:46.633173    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb"
Sep 11 11:39:46.653727 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:46.653661    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb" podStartSLOduration=3.369037674 podStartE2EDuration="7.653619326s" totalImagesPullingTime="3.196878703s" totalInitContainerRuntime="1.087702949s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:39:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:39:46.652406742 +0000 UTC m=+327.073012792" watchObservedRunningTime="2026-09-11 11:39:46.653619326 +0000 UTC m=+327.074225354"
Sep 11 11:39:49.170033 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:49.169999    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:49.230915 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:49.230872    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:39:57.475903 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:57.475866    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-77956949cf-bjlrz"]
Sep 11 11:39:57.484603 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:57.484580    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-77956949cf-bjlrz"
Sep 11 11:39:57.487498 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:57.487385    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 11 11:39:57.487498 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:57.487396    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 11 11:39:57.487498 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:57.487430    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 11 11:39:57.488107 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:57.488085    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-zhdth\""
Sep 11 11:39:57.488790 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:57.488771    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-77956949cf-bjlrz"]
Sep 11 11:39:57.529340 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:57.529318    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f1eb6de-66ef-49f8-a33f-363cc1a20412-metrics-cert\") pod \"lws-controller-manager-77956949cf-bjlrz\" (UID: \"9f1eb6de-66ef-49f8-a33f-363cc1a20412\") " pod="openshift-lws-operator/lws-controller-manager-77956949cf-bjlrz"
Sep 11 11:39:57.529440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:57.529359    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cljb5\" (UniqueName: \"kubernetes.io/projected/9f1eb6de-66ef-49f8-a33f-363cc1a20412-kube-api-access-cljb5\") pod \"lws-controller-manager-77956949cf-bjlrz\" (UID: \"9f1eb6de-66ef-49f8-a33f-363cc1a20412\") " pod="openshift-lws-operator/lws-controller-manager-77956949cf-bjlrz"
Sep 11 11:39:57.529440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:57.529379    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f1eb6de-66ef-49f8-a33f-363cc1a20412-cert\") pod \"lws-controller-manager-77956949cf-bjlrz\" (UID: \"9f1eb6de-66ef-49f8-a33f-363cc1a20412\") " pod="openshift-lws-operator/lws-controller-manager-77956949cf-bjlrz"
Sep 11 11:39:57.529515 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:57.529481    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9f1eb6de-66ef-49f8-a33f-363cc1a20412-manager-config\") pod \"lws-controller-manager-77956949cf-bjlrz\" (UID: \"9f1eb6de-66ef-49f8-a33f-363cc1a20412\") " pod="openshift-lws-operator/lws-controller-manager-77956949cf-bjlrz"
Sep 11 11:39:57.630028 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:57.630000    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f1eb6de-66ef-49f8-a33f-363cc1a20412-metrics-cert\") pod \"lws-controller-manager-77956949cf-bjlrz\" (UID: \"9f1eb6de-66ef-49f8-a33f-363cc1a20412\") " pod="openshift-lws-operator/lws-controller-manager-77956949cf-bjlrz"
Sep 11 11:39:57.630158 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:57.630068    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cljb5\" (UniqueName: \"kubernetes.io/projected/9f1eb6de-66ef-49f8-a33f-363cc1a20412-kube-api-access-cljb5\") pod \"lws-controller-manager-77956949cf-bjlrz\" (UID: \"9f1eb6de-66ef-49f8-a33f-363cc1a20412\") " pod="openshift-lws-operator/lws-controller-manager-77956949cf-bjlrz"
Sep 11 11:39:57.630158 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:57.630097    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f1eb6de-66ef-49f8-a33f-363cc1a20412-cert\") pod \"lws-controller-manager-77956949cf-bjlrz\" (UID: \"9f1eb6de-66ef-49f8-a33f-363cc1a20412\") " pod="openshift-lws-operator/lws-controller-manager-77956949cf-bjlrz"
Sep 11 11:39:57.630249 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:57.630156    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9f1eb6de-66ef-49f8-a33f-363cc1a20412-manager-config\") pod \"lws-controller-manager-77956949cf-bjlrz\" (UID: \"9f1eb6de-66ef-49f8-a33f-363cc1a20412\") " pod="openshift-lws-operator/lws-controller-manager-77956949cf-bjlrz"
Sep 11 11:39:57.630878 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:57.630804    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9f1eb6de-66ef-49f8-a33f-363cc1a20412-manager-config\") pod \"lws-controller-manager-77956949cf-bjlrz\" (UID: \"9f1eb6de-66ef-49f8-a33f-363cc1a20412\") " pod="openshift-lws-operator/lws-controller-manager-77956949cf-bjlrz"
Sep 11 11:39:57.632740 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:57.632721    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f1eb6de-66ef-49f8-a33f-363cc1a20412-cert\") pod \"lws-controller-manager-77956949cf-bjlrz\" (UID: \"9f1eb6de-66ef-49f8-a33f-363cc1a20412\") " pod="openshift-lws-operator/lws-controller-manager-77956949cf-bjlrz"
Sep 11 11:39:57.632832 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:57.632770    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f1eb6de-66ef-49f8-a33f-363cc1a20412-metrics-cert\") pod \"lws-controller-manager-77956949cf-bjlrz\" (UID: \"9f1eb6de-66ef-49f8-a33f-363cc1a20412\") " pod="openshift-lws-operator/lws-controller-manager-77956949cf-bjlrz"
Sep 11 11:39:57.639182 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:57.639160    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cljb5\" (UniqueName: \"kubernetes.io/projected/9f1eb6de-66ef-49f8-a33f-363cc1a20412-kube-api-access-cljb5\") pod \"lws-controller-manager-77956949cf-bjlrz\" (UID: \"9f1eb6de-66ef-49f8-a33f-363cc1a20412\") " pod="openshift-lws-operator/lws-controller-manager-77956949cf-bjlrz"
Sep 11 11:39:57.640218 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:57.640199    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-86f769d976-2s4jb"
Sep 11 11:39:57.794552 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:57.794528    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-77956949cf-bjlrz"
Sep 11 11:39:58.065222 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:58.065198    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-77956949cf-bjlrz"]
Sep 11 11:39:58.067574 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:39:58.067546    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f1eb6de_66ef_49f8_a33f_363cc1a20412.slice/crio-61cae436ccd39d9ad8ba4d96195c0d79beef01a382565d30b7e46c0b8a3caed0 WatchSource:0}: Error finding container 61cae436ccd39d9ad8ba4d96195c0d79beef01a382565d30b7e46c0b8a3caed0: Status 404 returned error can't find the container with id 61cae436ccd39d9ad8ba4d96195c0d79beef01a382565d30b7e46c0b8a3caed0
Sep 11 11:39:58.688055 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:58.688022    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-77956949cf-bjlrz" event={"ID":"9f1eb6de-66ef-49f8-a33f-363cc1a20412","Type":"ContainerStarted","Data":"61cae436ccd39d9ad8ba4d96195c0d79beef01a382565d30b7e46c0b8a3caed0"}
Sep 11 11:39:59.693874 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:59.693789    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-77956949cf-bjlrz" event={"ID":"9f1eb6de-66ef-49f8-a33f-363cc1a20412","Type":"ContainerStarted","Data":"5c6923e988445c7ef3374d90e8f6ba8f7e9b3687752f91c47bdf21b772add704"}
Sep 11 11:39:59.694225 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:59.693906    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-77956949cf-bjlrz"
Sep 11 11:39:59.729252 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:39:59.729207    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-77956949cf-bjlrz" podStartSLOduration=1.404173315 podStartE2EDuration="2.729192407s" totalImagesPullingTime="1.325019092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:39:59.728845635 +0000 UTC m=+340.149451662" watchObservedRunningTime="2026-09-11 11:39:59.729192407 +0000 UTC m=+340.149798464"
Sep 11 11:40:00.170343 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:00.170306    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:00.341520 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:00.341489    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:00.729803 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:00.729772    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:00.801432 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:00.801405    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-797c7d88fd-snkdh"]
Sep 11 11:40:00.804731 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:00.804712    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-797c7d88fd-snkdh"
Sep 11 11:40:00.807242 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:00.807220    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 11:40:00.808106 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:00.808090    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 11 11:40:00.808106 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:00.808096    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 11 11:40:00.808305 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:00.808205    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 11:40:00.808305 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:00.808214    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-62bkj\""
Sep 11 11:40:00.825332 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:00.825306    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-797c7d88fd-snkdh"]
Sep 11 11:40:00.863855 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:00.863829    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bda5443a-7bf7-435f-a397-cbf83459fb66-tls-certs\") pod \"kube-auth-proxy-797c7d88fd-snkdh\" (UID: \"bda5443a-7bf7-435f-a397-cbf83459fb66\") " pod="openshift-ingress/kube-auth-proxy-797c7d88fd-snkdh"
Sep 11 11:40:00.863966 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:00.863869    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bda5443a-7bf7-435f-a397-cbf83459fb66-tmp\") pod \"kube-auth-proxy-797c7d88fd-snkdh\" (UID: \"bda5443a-7bf7-435f-a397-cbf83459fb66\") " pod="openshift-ingress/kube-auth-proxy-797c7d88fd-snkdh"
Sep 11 11:40:00.863966 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:00.863885    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg5hg\" (UniqueName: \"kubernetes.io/projected/bda5443a-7bf7-435f-a397-cbf83459fb66-kube-api-access-cg5hg\") pod \"kube-auth-proxy-797c7d88fd-snkdh\" (UID: \"bda5443a-7bf7-435f-a397-cbf83459fb66\") " pod="openshift-ingress/kube-auth-proxy-797c7d88fd-snkdh"
Sep 11 11:40:00.876427 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:00.876395    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:00.965059 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:00.965026    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bda5443a-7bf7-435f-a397-cbf83459fb66-tls-certs\") pod \"kube-auth-proxy-797c7d88fd-snkdh\" (UID: \"bda5443a-7bf7-435f-a397-cbf83459fb66\") " pod="openshift-ingress/kube-auth-proxy-797c7d88fd-snkdh"
Sep 11 11:40:00.965249 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:00.965068    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bda5443a-7bf7-435f-a397-cbf83459fb66-tmp\") pod \"kube-auth-proxy-797c7d88fd-snkdh\" (UID: \"bda5443a-7bf7-435f-a397-cbf83459fb66\") " pod="openshift-ingress/kube-auth-proxy-797c7d88fd-snkdh"
Sep 11 11:40:00.965249 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:00.965085    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg5hg\" (UniqueName: \"kubernetes.io/projected/bda5443a-7bf7-435f-a397-cbf83459fb66-kube-api-access-cg5hg\") pod \"kube-auth-proxy-797c7d88fd-snkdh\" (UID: \"bda5443a-7bf7-435f-a397-cbf83459fb66\") " pod="openshift-ingress/kube-auth-proxy-797c7d88fd-snkdh"
Sep 11 11:40:00.967507 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:00.967485    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bda5443a-7bf7-435f-a397-cbf83459fb66-tmp\") pod \"kube-auth-proxy-797c7d88fd-snkdh\" (UID: \"bda5443a-7bf7-435f-a397-cbf83459fb66\") " pod="openshift-ingress/kube-auth-proxy-797c7d88fd-snkdh"
Sep 11 11:40:00.967655 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:00.967618    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bda5443a-7bf7-435f-a397-cbf83459fb66-tls-certs\") pod \"kube-auth-proxy-797c7d88fd-snkdh\" (UID: \"bda5443a-7bf7-435f-a397-cbf83459fb66\") " pod="openshift-ingress/kube-auth-proxy-797c7d88fd-snkdh"
Sep 11 11:40:00.984846 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:00.984822    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg5hg\" (UniqueName: \"kubernetes.io/projected/bda5443a-7bf7-435f-a397-cbf83459fb66-kube-api-access-cg5hg\") pod \"kube-auth-proxy-797c7d88fd-snkdh\" (UID: \"bda5443a-7bf7-435f-a397-cbf83459fb66\") " pod="openshift-ingress/kube-auth-proxy-797c7d88fd-snkdh"
Sep 11 11:40:01.113787 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:01.113759    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-797c7d88fd-snkdh"
Sep 11 11:40:01.387669 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:01.387624    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-797c7d88fd-snkdh"]
Sep 11 11:40:01.389507 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:40:01.389481    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbda5443a_7bf7_435f_a397_cbf83459fb66.slice/crio-72846e91429c71288c26bf375a3b912a46d8690add5890fe7c348701b6dbdef2 WatchSource:0}: Error finding container 72846e91429c71288c26bf375a3b912a46d8690add5890fe7c348701b6dbdef2: Status 404 returned error can't find the container with id 72846e91429c71288c26bf375a3b912a46d8690add5890fe7c348701b6dbdef2
Sep 11 11:40:01.703668 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:01.703568    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-797c7d88fd-snkdh" event={"ID":"bda5443a-7bf7-435f-a397-cbf83459fb66","Type":"ContainerStarted","Data":"72846e91429c71288c26bf375a3b912a46d8690add5890fe7c348701b6dbdef2"}
Sep 11 11:40:02.359180 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:02.359137    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:02.481384 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:02.481352    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:03.696871 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:03.696833    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:04.725858 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:04.725823    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-797c7d88fd-snkdh" event={"ID":"bda5443a-7bf7-435f-a397-cbf83459fb66","Type":"ContainerStarted","Data":"0afd18c43345d36f2fa93d18c79d3dbcd09d3c4e872576ff31ae31a591bd524a"}
Sep 11 11:40:04.744753 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:04.744711    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-797c7d88fd-snkdh" podStartSLOduration=1.564455785 podStartE2EDuration="4.744695327s" totalImagesPullingTime="3.180239542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:40:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:40:04.743720778 +0000 UTC m=+345.164326806" watchObservedRunningTime="2026-09-11 11:40:04.744695327 +0000 UTC m=+345.165301555"
Sep 11 11:40:10.700271 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:10.700241    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-77956949cf-bjlrz"
Sep 11 11:40:11.213819 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:11.213778    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:11.314262 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:11.314227    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:11.414262 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:11.414217    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:11.517958 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:11.517926    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:11.876245 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:11.876160    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:11.973741 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:11.973702    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:16.240577 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:16.240545    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:16.341219 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:16.341184    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:16.531256 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:16.531171    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:17.229420 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.229387    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:17.376231 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.376195    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:17.694171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.694137    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c66799df7-fb6z8"]
Sep 11 11:40:17.700254 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.700234    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:17.716703 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.716681    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c66799df7-fb6z8"]
Sep 11 11:40:17.766227 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.766190    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:17.814618 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.814587    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a10215c-ab9e-4c6c-a728-33503b6a2281-console-oauth-config\") pod \"console-6c66799df7-fb6z8\" (UID: \"7a10215c-ab9e-4c6c-a728-33503b6a2281\") " pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:17.814789 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.814657    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a10215c-ab9e-4c6c-a728-33503b6a2281-trusted-ca-bundle\") pod \"console-6c66799df7-fb6z8\" (UID: \"7a10215c-ab9e-4c6c-a728-33503b6a2281\") " pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:17.814789 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.814695    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a10215c-ab9e-4c6c-a728-33503b6a2281-oauth-serving-cert\") pod \"console-6c66799df7-fb6z8\" (UID: \"7a10215c-ab9e-4c6c-a728-33503b6a2281\") " pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:17.814789 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.814722    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phl97\" (UniqueName: \"kubernetes.io/projected/7a10215c-ab9e-4c6c-a728-33503b6a2281-kube-api-access-phl97\") pod \"console-6c66799df7-fb6z8\" (UID: \"7a10215c-ab9e-4c6c-a728-33503b6a2281\") " pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:17.814789 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.814743    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a10215c-ab9e-4c6c-a728-33503b6a2281-service-ca\") pod \"console-6c66799df7-fb6z8\" (UID: \"7a10215c-ab9e-4c6c-a728-33503b6a2281\") " pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:17.814952 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.814851    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a10215c-ab9e-4c6c-a728-33503b6a2281-console-serving-cert\") pod \"console-6c66799df7-fb6z8\" (UID: \"7a10215c-ab9e-4c6c-a728-33503b6a2281\") " pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:17.814952 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.814877    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a10215c-ab9e-4c6c-a728-33503b6a2281-console-config\") pod \"console-6c66799df7-fb6z8\" (UID: \"7a10215c-ab9e-4c6c-a728-33503b6a2281\") " pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:17.916077 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.916042    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a10215c-ab9e-4c6c-a728-33503b6a2281-console-oauth-config\") pod \"console-6c66799df7-fb6z8\" (UID: \"7a10215c-ab9e-4c6c-a728-33503b6a2281\") " pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:17.916234 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.916086    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a10215c-ab9e-4c6c-a728-33503b6a2281-trusted-ca-bundle\") pod \"console-6c66799df7-fb6z8\" (UID: \"7a10215c-ab9e-4c6c-a728-33503b6a2281\") " pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:17.916234 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.916105    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a10215c-ab9e-4c6c-a728-33503b6a2281-oauth-serving-cert\") pod \"console-6c66799df7-fb6z8\" (UID: \"7a10215c-ab9e-4c6c-a728-33503b6a2281\") " pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:17.916234 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.916135    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phl97\" (UniqueName: \"kubernetes.io/projected/7a10215c-ab9e-4c6c-a728-33503b6a2281-kube-api-access-phl97\") pod \"console-6c66799df7-fb6z8\" (UID: \"7a10215c-ab9e-4c6c-a728-33503b6a2281\") " pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:17.916234 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.916167    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a10215c-ab9e-4c6c-a728-33503b6a2281-service-ca\") pod \"console-6c66799df7-fb6z8\" (UID: \"7a10215c-ab9e-4c6c-a728-33503b6a2281\") " pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:17.916434 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.916280    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a10215c-ab9e-4c6c-a728-33503b6a2281-console-serving-cert\") pod \"console-6c66799df7-fb6z8\" (UID: \"7a10215c-ab9e-4c6c-a728-33503b6a2281\") " pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:17.916434 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.916312    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a10215c-ab9e-4c6c-a728-33503b6a2281-console-config\") pod \"console-6c66799df7-fb6z8\" (UID: \"7a10215c-ab9e-4c6c-a728-33503b6a2281\") " pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:17.917058 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.917029    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a10215c-ab9e-4c6c-a728-33503b6a2281-console-config\") pod \"console-6c66799df7-fb6z8\" (UID: \"7a10215c-ab9e-4c6c-a728-33503b6a2281\") " pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:17.917200 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.917120    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a10215c-ab9e-4c6c-a728-33503b6a2281-oauth-serving-cert\") pod \"console-6c66799df7-fb6z8\" (UID: \"7a10215c-ab9e-4c6c-a728-33503b6a2281\") " pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:17.917200 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.917127    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a10215c-ab9e-4c6c-a728-33503b6a2281-service-ca\") pod \"console-6c66799df7-fb6z8\" (UID: \"7a10215c-ab9e-4c6c-a728-33503b6a2281\") " pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:17.917200 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.917169    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a10215c-ab9e-4c6c-a728-33503b6a2281-trusted-ca-bundle\") pod \"console-6c66799df7-fb6z8\" (UID: \"7a10215c-ab9e-4c6c-a728-33503b6a2281\") " pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:17.918787 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.918760    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a10215c-ab9e-4c6c-a728-33503b6a2281-console-serving-cert\") pod \"console-6c66799df7-fb6z8\" (UID: \"7a10215c-ab9e-4c6c-a728-33503b6a2281\") " pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:17.918877 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.918847    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a10215c-ab9e-4c6c-a728-33503b6a2281-console-oauth-config\") pod \"console-6c66799df7-fb6z8\" (UID: \"7a10215c-ab9e-4c6c-a728-33503b6a2281\") " pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:17.920259 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.920237    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:17.931757 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:17.931728    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phl97\" (UniqueName: \"kubernetes.io/projected/7a10215c-ab9e-4c6c-a728-33503b6a2281-kube-api-access-phl97\") pod \"console-6c66799df7-fb6z8\" (UID: \"7a10215c-ab9e-4c6c-a728-33503b6a2281\") " pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:18.009452 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:18.009424    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:18.290723 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:18.290492    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c66799df7-fb6z8"]
Sep 11 11:40:18.293264 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:40:18.293221    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a10215c_ab9e_4c6c_a728_33503b6a2281.slice/crio-c5d99cd48a8e35d37b623d5f34da2fa3d57d2e0aa4d2633b65c7ab68ba00bddb WatchSource:0}: Error finding container c5d99cd48a8e35d37b623d5f34da2fa3d57d2e0aa4d2633b65c7ab68ba00bddb: Status 404 returned error can't find the container with id c5d99cd48a8e35d37b623d5f34da2fa3d57d2e0aa4d2633b65c7ab68ba00bddb
Sep 11 11:40:18.796833 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:18.796800    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c66799df7-fb6z8" event={"ID":"7a10215c-ab9e-4c6c-a728-33503b6a2281","Type":"ContainerStarted","Data":"aacf24499538f6471e42efd7b8032b4de69603aef8ac4aa720e827636c931f70"}
Sep 11 11:40:18.797207 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:18.796854    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c66799df7-fb6z8" event={"ID":"7a10215c-ab9e-4c6c-a728-33503b6a2281","Type":"ContainerStarted","Data":"c5d99cd48a8e35d37b623d5f34da2fa3d57d2e0aa4d2633b65c7ab68ba00bddb"}
Sep 11 11:40:18.822420 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:18.822367    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c66799df7-fb6z8" podStartSLOduration=1.822350184 podStartE2EDuration="1.822350184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:40:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:40:18.821896879 +0000 UTC m=+359.242502908" watchObservedRunningTime="2026-09-11 11:40:18.822350184 +0000 UTC m=+359.242956252"
Sep 11 11:40:21.342346 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:21.342313    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:21.477572 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:21.477534    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:21.635564 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:21.635475    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:21.733248 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:21.733220    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:21.954249 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:21.954150    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:22.774494 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:22.774458    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:22.857080 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:22.857046    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:28.009991 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:28.009959    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:28.010431 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:28.010043    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:28.014719 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:28.014696    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:28.844426 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:28.844399    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c66799df7-fb6z8"
Sep 11 11:40:28.895237 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:28.895206    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b44699f76-7pcsz"]
Sep 11 11:40:29.012161 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:29.012133    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:29.107986 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:29.107898    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:29.197759 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:29.197724    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:37.150498 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:37.150451    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:39.225493 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:39.225461    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:39.312679 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:39.312651    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:40:53.915988 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:53.915945    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-b44699f76-7pcsz" podUID="fe149968-7812-4489-960d-c070b29e300b" containerName="console" containerID="cri-o://7b995bab0567804f9652b18e44109582786f38db59f3b6e310dc5344b3b610e1" gracePeriod=15
Sep 11 11:40:53.921475 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:53.921441    2575 patch_prober.go:28] interesting pod/console-b44699f76-7pcsz container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.134.0.22:8443/health\": dial tcp 10.134.0.22:8443: connect: connection refused" start-of-body=
Sep 11 11:40:53.921599 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:53.921506    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-b44699f76-7pcsz" podUID="fe149968-7812-4489-960d-c070b29e300b" containerName="console" probeResult="failure" output="Get \"https://10.134.0.22:8443/health\": dial tcp 10.134.0.22:8443: connect: connection refused"
Sep 11 11:40:54.160941 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.160916    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:40:54.237152 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.237127    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-oauth-serving-cert\") pod \"fe149968-7812-4489-960d-c070b29e300b\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") "
Sep 11 11:40:54.237299 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.237233    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe149968-7812-4489-960d-c070b29e300b-console-oauth-config\") pod \"fe149968-7812-4489-960d-c070b29e300b\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") "
Sep 11 11:40:54.237299 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.237264    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-service-ca\") pod \"fe149968-7812-4489-960d-c070b29e300b\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") "
Sep 11 11:40:54.237299 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.237291    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-console-config\") pod \"fe149968-7812-4489-960d-c070b29e300b\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") "
Sep 11 11:40:54.237475 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.237325    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe149968-7812-4489-960d-c070b29e300b-console-serving-cert\") pod \"fe149968-7812-4489-960d-c070b29e300b\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") "
Sep 11 11:40:54.237475 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.237408    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-trusted-ca-bundle\") pod \"fe149968-7812-4489-960d-c070b29e300b\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") "
Sep 11 11:40:54.237475 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.237432    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfxzh\" (UniqueName: \"kubernetes.io/projected/fe149968-7812-4489-960d-c070b29e300b-kube-api-access-jfxzh\") pod \"fe149968-7812-4489-960d-c070b29e300b\" (UID: \"fe149968-7812-4489-960d-c070b29e300b\") "
Sep 11 11:40:54.237714 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.237567    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fe149968-7812-4489-960d-c070b29e300b" (UID: "fe149968-7812-4489-960d-c070b29e300b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:40:54.237815 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.237757    2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-oauth-serving-cert\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:40:54.237815 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.237761    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-service-ca" (OuterVolumeSpecName: "service-ca") pod "fe149968-7812-4489-960d-c070b29e300b" (UID: "fe149968-7812-4489-960d-c070b29e300b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:40:54.237918 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.237812    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-console-config" (OuterVolumeSpecName: "console-config") pod "fe149968-7812-4489-960d-c070b29e300b" (UID: "fe149968-7812-4489-960d-c070b29e300b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:40:54.237918 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.237871    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fe149968-7812-4489-960d-c070b29e300b" (UID: "fe149968-7812-4489-960d-c070b29e300b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:40:54.239595 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.239573    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe149968-7812-4489-960d-c070b29e300b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fe149968-7812-4489-960d-c070b29e300b" (UID: "fe149968-7812-4489-960d-c070b29e300b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:40:54.239743 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.239598    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe149968-7812-4489-960d-c070b29e300b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fe149968-7812-4489-960d-c070b29e300b" (UID: "fe149968-7812-4489-960d-c070b29e300b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:40:54.239743 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.239615    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe149968-7812-4489-960d-c070b29e300b-kube-api-access-jfxzh" (OuterVolumeSpecName: "kube-api-access-jfxzh") pod "fe149968-7812-4489-960d-c070b29e300b" (UID: "fe149968-7812-4489-960d-c070b29e300b"). InnerVolumeSpecName "kube-api-access-jfxzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:40:54.338913 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.338879    2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe149968-7812-4489-960d-c070b29e300b-console-serving-cert\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:40:54.338913 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.338906    2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-trusted-ca-bundle\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:40:54.338913 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.338915    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfxzh\" (UniqueName: \"kubernetes.io/projected/fe149968-7812-4489-960d-c070b29e300b-kube-api-access-jfxzh\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:40:54.339131 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.338923    2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe149968-7812-4489-960d-c070b29e300b-console-oauth-config\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:40:54.339131 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.338934    2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-service-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:40:54.339131 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.338944    2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe149968-7812-4489-960d-c070b29e300b-console-config\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:40:54.961814 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.961784    2575 generic.go:356] "Generic (PLEG): container finished" podID="fe149968-7812-4489-960d-c070b29e300b" containerID="7b995bab0567804f9652b18e44109582786f38db59f3b6e310dc5344b3b610e1" exitCode=0
Sep 11 11:40:54.962222 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.961827    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b44699f76-7pcsz" event={"ID":"fe149968-7812-4489-960d-c070b29e300b","Type":"ContainerDied","Data":"7b995bab0567804f9652b18e44109582786f38db59f3b6e310dc5344b3b610e1"}
Sep 11 11:40:54.962222 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.961863    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b44699f76-7pcsz" event={"ID":"fe149968-7812-4489-960d-c070b29e300b","Type":"ContainerDied","Data":"bc8ca6aa3c28e43dd883695f1f2155c443707bce1a90ef55a7d91b7b88f99491"}
Sep 11 11:40:54.962222 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.961883    2575 scope.go:117] "RemoveContainer" containerID="7b995bab0567804f9652b18e44109582786f38db59f3b6e310dc5344b3b610e1"
Sep 11 11:40:54.962222 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.961830    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b44699f76-7pcsz"
Sep 11 11:40:54.970902 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.970882    2575 scope.go:117] "RemoveContainer" containerID="7b995bab0567804f9652b18e44109582786f38db59f3b6e310dc5344b3b610e1"
Sep 11 11:40:54.971152 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:40:54.971133    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b995bab0567804f9652b18e44109582786f38db59f3b6e310dc5344b3b610e1\": container with ID starting with 7b995bab0567804f9652b18e44109582786f38db59f3b6e310dc5344b3b610e1 not found: ID does not exist" containerID="7b995bab0567804f9652b18e44109582786f38db59f3b6e310dc5344b3b610e1"
Sep 11 11:40:54.971211 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.971164    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b995bab0567804f9652b18e44109582786f38db59f3b6e310dc5344b3b610e1"} err="failed to get container status \"7b995bab0567804f9652b18e44109582786f38db59f3b6e310dc5344b3b610e1\": rpc error: code = NotFound desc = could not find container \"7b995bab0567804f9652b18e44109582786f38db59f3b6e310dc5344b3b610e1\": container with ID starting with 7b995bab0567804f9652b18e44109582786f38db59f3b6e310dc5344b3b610e1 not found: ID does not exist"
Sep 11 11:40:54.989448 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.989424    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b44699f76-7pcsz"]
Sep 11 11:40:54.991528 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:54.991509    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b44699f76-7pcsz"]
Sep 11 11:40:56.081007 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:40:56.080974    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe149968-7812-4489-960d-c070b29e300b" path="/var/lib/kubelet/pods/fe149968-7812-4489-960d-c070b29e300b/volumes"
Sep 11 11:41:12.350448 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:12.350414    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:12.535351 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:12.535321    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:24.110814 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:24.110770    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:36.235074 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:36.235033    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:37.031016 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:37.030989    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:40.269403 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:40.269350    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:40.531441 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:40.531353    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:40.681828 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:40.681797    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:41.940337 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:41.940253    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:42.640534 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:42.640493    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:43.273915 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:43.273876    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:44.685333 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:44.685295    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:45.088833 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:45.088805    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:45.487864 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:45.487830    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:45.984791 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:45.984751    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:46.148553 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:46.148508    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:48.582273 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:48.582239    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7"]
Sep 11 11:41:48.582668 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:48.582579    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe149968-7812-4489-960d-c070b29e300b" containerName="console"
Sep 11 11:41:48.582668 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:48.582590    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe149968-7812-4489-960d-c070b29e300b" containerName="console"
Sep 11 11:41:48.582770 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:48.582685    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe149968-7812-4489-960d-c070b29e300b" containerName="console"
Sep 11 11:41:48.585670 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:48.585651    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7"
Sep 11 11:41:48.591388 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:48.591360    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 11:41:48.591526 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:48.591369    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 11:41:48.591607 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:48.591589    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8dpsl\""
Sep 11 11:41:48.618385 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:48.618358    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7"]
Sep 11 11:41:48.715531 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:48.715494    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn9c7\" (UniqueName: \"kubernetes.io/projected/d7c05453-1388-4d12-943b-074a51aa7866-kube-api-access-sn9c7\") pod \"kuadrant-operator-controller-manager-76f8f686bf-b22r7\" (UID: \"d7c05453-1388-4d12-943b-074a51aa7866\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7"
Sep 11 11:41:48.715721 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:48.715628    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7c05453-1388-4d12-943b-074a51aa7866-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-b22r7\" (UID: \"d7c05453-1388-4d12-943b-074a51aa7866\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7"
Sep 11 11:41:48.816361 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:48.816329    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn9c7\" (UniqueName: \"kubernetes.io/projected/d7c05453-1388-4d12-943b-074a51aa7866-kube-api-access-sn9c7\") pod \"kuadrant-operator-controller-manager-76f8f686bf-b22r7\" (UID: \"d7c05453-1388-4d12-943b-074a51aa7866\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7"
Sep 11 11:41:48.816483 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:48.816416    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7c05453-1388-4d12-943b-074a51aa7866-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-b22r7\" (UID: \"d7c05453-1388-4d12-943b-074a51aa7866\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7"
Sep 11 11:41:48.816772 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:48.816755    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7c05453-1388-4d12-943b-074a51aa7866-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-b22r7\" (UID: \"d7c05453-1388-4d12-943b-074a51aa7866\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7"
Sep 11 11:41:48.825277 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:48.825252    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn9c7\" (UniqueName: \"kubernetes.io/projected/d7c05453-1388-4d12-943b-074a51aa7866-kube-api-access-sn9c7\") pod \"kuadrant-operator-controller-manager-76f8f686bf-b22r7\" (UID: \"d7c05453-1388-4d12-943b-074a51aa7866\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7"
Sep 11 11:41:48.896593 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:48.896521    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7"
Sep 11 11:41:49.193798 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:49.193450    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7"]
Sep 11 11:41:49.195977 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:41:49.195943    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7c05453_1388_4d12_943b_074a51aa7866.slice/crio-922d47cb5c4a2c44f247ef5ace98b94a6e4cc81f4623ab60a3cdae1c471209a5 WatchSource:0}: Error finding container 922d47cb5c4a2c44f247ef5ace98b94a6e4cc81f4623ab60a3cdae1c471209a5: Status 404 returned error can't find the container with id 922d47cb5c4a2c44f247ef5ace98b94a6e4cc81f4623ab60a3cdae1c471209a5
Sep 11 11:41:49.210279 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:49.210248    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7" event={"ID":"d7c05453-1388-4d12-943b-074a51aa7866","Type":"ContainerStarted","Data":"922d47cb5c4a2c44f247ef5ace98b94a6e4cc81f4623ab60a3cdae1c471209a5"}
Sep 11 11:41:50.042560 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:50.042506    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:50.148237 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:50.148200    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:50.649974 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:50.649934    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:50.789600 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:50.789568    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:53.049018 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:53.048990    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:53.147618 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:53.147581    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:53.236837 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:53.236810    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7" event={"ID":"d7c05453-1388-4d12-943b-074a51aa7866","Type":"ContainerStarted","Data":"9865e579e6dcceab1ed2c6e7f9288af9bfcfd349c5281724d7bf45581b9cd8c2"}
Sep 11 11:41:53.237002 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:53.236989    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7"
Sep 11 11:41:53.260143 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:53.260103    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7" podStartSLOduration=1.29755165 podStartE2EDuration="5.260087796s" totalImagesPullingTime="3.962536146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:41:53.258052604 +0000 UTC m=+453.678658630" watchObservedRunningTime="2026-09-11 11:41:53.260087796 +0000 UTC m=+453.680693820"
Sep 11 11:41:55.948264 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:55.948222    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:56.063103 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:56.063060    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:58.093955 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:58.093923    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:41:58.196953 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:41:58.196919    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:42:03.034672 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:03.034612    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:42:03.099877 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:03.099841    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:42:03.454799 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:03.454705    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:42:03.657626 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:03.657583    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:42:03.848529 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:03.848494    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:42:03.918428 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:03.918394    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:42:04.246322 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:04.246296    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7"
Sep 11 11:42:06.564236 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:06.564200    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7"]
Sep 11 11:42:06.564662 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:06.564388    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7" podUID="d7c05453-1388-4d12-943b-074a51aa7866" containerName="manager" containerID="cri-o://9865e579e6dcceab1ed2c6e7f9288af9bfcfd349c5281724d7bf45581b9cd8c2" gracePeriod=2
Sep 11 11:42:06.578081 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:06.578058    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7"]
Sep 11 11:42:06.584373 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:42:06.584330    2575 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d7c05453-1388-4d12-943b-074a51aa7866\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-b22r7\": pods \"kuadrant-operator-controller-manager-76f8f686bf-b22r7\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7"
Sep 11 11:42:06.596974 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:06.596948    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt"]
Sep 11 11:42:06.597497 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:06.597475    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7c05453-1388-4d12-943b-074a51aa7866" containerName="manager"
Sep 11 11:42:06.597656 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:06.597562    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7c05453-1388-4d12-943b-074a51aa7866" containerName="manager"
Sep 11 11:42:06.597745 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:06.597703    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7c05453-1388-4d12-943b-074a51aa7866" containerName="manager"
Sep 11 11:42:06.600883 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:06.600865    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt"
Sep 11 11:42:06.625813 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:06.625780    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt"]
Sep 11 11:42:06.663700 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:42:06.663663    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-b22r7\" is forbidden: User \"system:node:ip-10-0-135-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-16.ec2.internal' and this object" podUID="d7c05453-1388-4d12-943b-074a51aa7866" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7"
Sep 11 11:42:06.683816 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:06.683790    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz7v2\" (UniqueName: \"kubernetes.io/projected/dc798d41-7579-4610-960f-b869086ce0a6-kube-api-access-rz7v2\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jsbgt\" (UID: \"dc798d41-7579-4610-960f-b869086ce0a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt"
Sep 11 11:42:06.683891 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:06.683874    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc798d41-7579-4610-960f-b869086ce0a6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jsbgt\" (UID: \"dc798d41-7579-4610-960f-b869086ce0a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt"
Sep 11 11:42:06.784517 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:06.784491    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc798d41-7579-4610-960f-b869086ce0a6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jsbgt\" (UID: \"dc798d41-7579-4610-960f-b869086ce0a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt"
Sep 11 11:42:06.784613 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:06.784576    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rz7v2\" (UniqueName: \"kubernetes.io/projected/dc798d41-7579-4610-960f-b869086ce0a6-kube-api-access-rz7v2\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jsbgt\" (UID: \"dc798d41-7579-4610-960f-b869086ce0a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt"
Sep 11 11:42:06.784900 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:06.784882    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc798d41-7579-4610-960f-b869086ce0a6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jsbgt\" (UID: \"dc798d41-7579-4610-960f-b869086ce0a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt"
Sep 11 11:42:06.795396 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:06.795369    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz7v2\" (UniqueName: \"kubernetes.io/projected/dc798d41-7579-4610-960f-b869086ce0a6-kube-api-access-rz7v2\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jsbgt\" (UID: \"dc798d41-7579-4610-960f-b869086ce0a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt"
Sep 11 11:42:06.810425 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:06.810406    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7"
Sep 11 11:42:06.812706 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:42:06.812684    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-b22r7\" is forbidden: User \"system:node:ip-10-0-135-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-16.ec2.internal' and this object" podUID="d7c05453-1388-4d12-943b-074a51aa7866" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7"
Sep 11 11:42:06.884974 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:06.884902    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sn9c7\" (UniqueName: \"kubernetes.io/projected/d7c05453-1388-4d12-943b-074a51aa7866-kube-api-access-sn9c7\") pod \"d7c05453-1388-4d12-943b-074a51aa7866\" (UID: \"d7c05453-1388-4d12-943b-074a51aa7866\") "
Sep 11 11:42:06.884974 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:06.884956    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7c05453-1388-4d12-943b-074a51aa7866-extensions-socket-volume\") pod \"d7c05453-1388-4d12-943b-074a51aa7866\" (UID: \"d7c05453-1388-4d12-943b-074a51aa7866\") "
Sep 11 11:42:06.885431 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:06.885409    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7c05453-1388-4d12-943b-074a51aa7866-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d7c05453-1388-4d12-943b-074a51aa7866" (UID: "d7c05453-1388-4d12-943b-074a51aa7866"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:42:06.887180 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:06.887153    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7c05453-1388-4d12-943b-074a51aa7866-kube-api-access-sn9c7" (OuterVolumeSpecName: "kube-api-access-sn9c7") pod "d7c05453-1388-4d12-943b-074a51aa7866" (UID: "d7c05453-1388-4d12-943b-074a51aa7866"). InnerVolumeSpecName "kube-api-access-sn9c7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:42:06.939098 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:06.939060    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt"
Sep 11 11:42:06.985980 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:06.985947    2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d7c05453-1388-4d12-943b-074a51aa7866-extensions-socket-volume\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:42:06.986090 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:06.985987    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sn9c7\" (UniqueName: \"kubernetes.io/projected/d7c05453-1388-4d12-943b-074a51aa7866-kube-api-access-sn9c7\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:42:07.220462 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:07.220347    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt"]
Sep 11 11:42:07.223413 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:42:07.223388    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc798d41_7579_4610_960f_b869086ce0a6.slice/crio-cc44b5308f3cc7c52925ab933300ebf9286fd46db2047121c4b92550e9ea4ef0 WatchSource:0}: Error finding container cc44b5308f3cc7c52925ab933300ebf9286fd46db2047121c4b92550e9ea4ef0: Status 404 returned error can't find the container with id cc44b5308f3cc7c52925ab933300ebf9286fd46db2047121c4b92550e9ea4ef0
Sep 11 11:42:07.306202 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:07.306173    2575 generic.go:356] "Generic (PLEG): container finished" podID="d7c05453-1388-4d12-943b-074a51aa7866" containerID="9865e579e6dcceab1ed2c6e7f9288af9bfcfd349c5281724d7bf45581b9cd8c2" exitCode=0
Sep 11 11:42:07.306364 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:07.306225    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7"
Sep 11 11:42:07.306364 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:07.306280    2575 scope.go:117] "RemoveContainer" containerID="9865e579e6dcceab1ed2c6e7f9288af9bfcfd349c5281724d7bf45581b9cd8c2"
Sep 11 11:42:07.308307 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:07.308280    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt" event={"ID":"dc798d41-7579-4610-960f-b869086ce0a6","Type":"ContainerStarted","Data":"ca17466d6e6c38f23a46ede80db0ec67cf275b624625d7666f207fb6707a7566"}
Sep 11 11:42:07.308417 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:07.308326    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt" event={"ID":"dc798d41-7579-4610-960f-b869086ce0a6","Type":"ContainerStarted","Data":"cc44b5308f3cc7c52925ab933300ebf9286fd46db2047121c4b92550e9ea4ef0"}
Sep 11 11:42:07.308481 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:07.308446    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt"
Sep 11 11:42:07.308703 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:42:07.308674    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-b22r7\" is forbidden: User \"system:node:ip-10-0-135-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-16.ec2.internal' and this object" podUID="d7c05453-1388-4d12-943b-074a51aa7866" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7"
Sep 11 11:42:07.315080 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:07.315066    2575 scope.go:117] "RemoveContainer" containerID="9865e579e6dcceab1ed2c6e7f9288af9bfcfd349c5281724d7bf45581b9cd8c2"
Sep 11 11:42:07.315335 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:42:07.315316    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9865e579e6dcceab1ed2c6e7f9288af9bfcfd349c5281724d7bf45581b9cd8c2\": container with ID starting with 9865e579e6dcceab1ed2c6e7f9288af9bfcfd349c5281724d7bf45581b9cd8c2 not found: ID does not exist" containerID="9865e579e6dcceab1ed2c6e7f9288af9bfcfd349c5281724d7bf45581b9cd8c2"
Sep 11 11:42:07.315390 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:07.315342    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9865e579e6dcceab1ed2c6e7f9288af9bfcfd349c5281724d7bf45581b9cd8c2"} err="failed to get container status \"9865e579e6dcceab1ed2c6e7f9288af9bfcfd349c5281724d7bf45581b9cd8c2\": rpc error: code = NotFound desc = could not find container \"9865e579e6dcceab1ed2c6e7f9288af9bfcfd349c5281724d7bf45581b9cd8c2\": container with ID starting with 9865e579e6dcceab1ed2c6e7f9288af9bfcfd349c5281724d7bf45581b9cd8c2 not found: ID does not exist"
Sep 11 11:42:07.339549 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:07.339505    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt" podStartSLOduration=1.33949155 podStartE2EDuration="1.33949155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:42:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:42:07.33788226 +0000 UTC m=+467.758488292" watchObservedRunningTime="2026-09-11 11:42:07.33949155 +0000 UTC m=+467.760097576"
Sep 11 11:42:07.339819 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:42:07.339796    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-b22r7\" is forbidden: User \"system:node:ip-10-0-135-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-16.ec2.internal' and this object" podUID="d7c05453-1388-4d12-943b-074a51aa7866" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7"
Sep 11 11:42:07.342950 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:42:07.342929    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-b22r7\" is forbidden: User \"system:node:ip-10-0-135-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-16.ec2.internal' and this object" podUID="d7c05453-1388-4d12-943b-074a51aa7866" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b22r7"
Sep 11 11:42:08.081946 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:08.081909    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7c05453-1388-4d12-943b-074a51aa7866" path="/var/lib/kubelet/pods/d7c05453-1388-4d12-943b-074a51aa7866/volumes"
Sep 11 11:42:10.026008 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:10.025980    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:42:10.135897 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:10.135868    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:42:13.509231 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:13.509196    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:42:13.643856 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:13.643827    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:42:17.485378 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:17.485333    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:42:17.587025 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:17.586986    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:42:18.316551 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:18.316522    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt"
Sep 11 11:42:22.357659 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:22.357594    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:42:22.475183 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:22.475154    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:42:22.506995 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:22.506958    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt"]
Sep 11 11:42:22.507227 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:22.507191    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt" podUID="dc798d41-7579-4610-960f-b869086ce0a6" containerName="manager" containerID="cri-o://ca17466d6e6c38f23a46ede80db0ec67cf275b624625d7666f207fb6707a7566" gracePeriod=10
Sep 11 11:42:22.589483 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:22.589448    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:42:22.792649 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:22.792617    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt"
Sep 11 11:42:22.821744 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:22.821721    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc798d41-7579-4610-960f-b869086ce0a6-extensions-socket-volume\") pod \"dc798d41-7579-4610-960f-b869086ce0a6\" (UID: \"dc798d41-7579-4610-960f-b869086ce0a6\") "
Sep 11 11:42:22.821892 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:22.821770    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rz7v2\" (UniqueName: \"kubernetes.io/projected/dc798d41-7579-4610-960f-b869086ce0a6-kube-api-access-rz7v2\") pod \"dc798d41-7579-4610-960f-b869086ce0a6\" (UID: \"dc798d41-7579-4610-960f-b869086ce0a6\") "
Sep 11 11:42:22.822072 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:22.822049    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc798d41-7579-4610-960f-b869086ce0a6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dc798d41-7579-4610-960f-b869086ce0a6" (UID: "dc798d41-7579-4610-960f-b869086ce0a6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:42:22.823954 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:22.823930    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc798d41-7579-4610-960f-b869086ce0a6-kube-api-access-rz7v2" (OuterVolumeSpecName: "kube-api-access-rz7v2") pod "dc798d41-7579-4610-960f-b869086ce0a6" (UID: "dc798d41-7579-4610-960f-b869086ce0a6"). InnerVolumeSpecName "kube-api-access-rz7v2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:42:22.922857 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:22.922771    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rz7v2\" (UniqueName: \"kubernetes.io/projected/dc798d41-7579-4610-960f-b869086ce0a6-kube-api-access-rz7v2\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:42:22.922857 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:22.922799    2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc798d41-7579-4610-960f-b869086ce0a6-extensions-socket-volume\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:42:23.383543 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:23.383504    2575 generic.go:356] "Generic (PLEG): container finished" podID="dc798d41-7579-4610-960f-b869086ce0a6" containerID="ca17466d6e6c38f23a46ede80db0ec67cf275b624625d7666f207fb6707a7566" exitCode=0
Sep 11 11:42:23.383975 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:23.383552    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt"
Sep 11 11:42:23.383975 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:23.383595    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt" event={"ID":"dc798d41-7579-4610-960f-b869086ce0a6","Type":"ContainerDied","Data":"ca17466d6e6c38f23a46ede80db0ec67cf275b624625d7666f207fb6707a7566"}
Sep 11 11:42:23.383975 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:23.383665    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt" event={"ID":"dc798d41-7579-4610-960f-b869086ce0a6","Type":"ContainerDied","Data":"cc44b5308f3cc7c52925ab933300ebf9286fd46db2047121c4b92550e9ea4ef0"}
Sep 11 11:42:23.383975 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:23.383694    2575 scope.go:117] "RemoveContainer" containerID="ca17466d6e6c38f23a46ede80db0ec67cf275b624625d7666f207fb6707a7566"
Sep 11 11:42:23.392535 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:23.392520    2575 scope.go:117] "RemoveContainer" containerID="ca17466d6e6c38f23a46ede80db0ec67cf275b624625d7666f207fb6707a7566"
Sep 11 11:42:23.392839 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:42:23.392821    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca17466d6e6c38f23a46ede80db0ec67cf275b624625d7666f207fb6707a7566\": container with ID starting with ca17466d6e6c38f23a46ede80db0ec67cf275b624625d7666f207fb6707a7566 not found: ID does not exist" containerID="ca17466d6e6c38f23a46ede80db0ec67cf275b624625d7666f207fb6707a7566"
Sep 11 11:42:23.392902 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:23.392848    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca17466d6e6c38f23a46ede80db0ec67cf275b624625d7666f207fb6707a7566"} err="failed to get container status \"ca17466d6e6c38f23a46ede80db0ec67cf275b624625d7666f207fb6707a7566\": rpc error: code = NotFound desc = could not find container \"ca17466d6e6c38f23a46ede80db0ec67cf275b624625d7666f207fb6707a7566\": container with ID starting with ca17466d6e6c38f23a46ede80db0ec67cf275b624625d7666f207fb6707a7566 not found: ID does not exist"
Sep 11 11:42:23.408242 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:23.408215    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt"]
Sep 11 11:42:23.415039 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:23.415012    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jsbgt"]
Sep 11 11:42:24.081035 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:24.081005    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc798d41-7579-4610-960f-b869086ce0a6" path="/var/lib/kubelet/pods/dc798d41-7579-4610-960f-b869086ce0a6/volumes"
Sep 11 11:42:39.002877 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:39.002843    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:42:39.164599 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:39.164567    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:42:51.426627 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:51.426594    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:42:53.648537 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:53.648501    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:42:53.795201 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:53.795165    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:42:59.977318 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:42:59.977281    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:43:00.003691 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:00.003658    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-b5z2x"]
Sep 11 11:43:00.004090 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:00.004072    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc798d41-7579-4610-960f-b869086ce0a6" containerName="manager"
Sep 11 11:43:00.004177 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:00.004093    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc798d41-7579-4610-960f-b869086ce0a6" containerName="manager"
Sep 11 11:43:00.004233 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:00.004195    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc798d41-7579-4610-960f-b869086ce0a6" containerName="manager"
Sep 11 11:43:00.007291 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:00.007271    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-b5z2x"
Sep 11 11:43:00.012216 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:00.012194    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 11 11:43:00.012216 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:00.012213    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 11:43:00.013079 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:00.013059    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 11:43:00.013161 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:00.013063    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4nnwd\""
Sep 11 11:43:00.021237 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:00.021214    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-b5z2x"]
Sep 11 11:43:00.038507 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:00.038478    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3eb2aba2-8a29-4aa5-b440-3107feab2c7c-config-file\") pod \"limitador-limitador-8588749977-b5z2x\" (UID: \"3eb2aba2-8a29-4aa5-b440-3107feab2c7c\") " pod="kuadrant-system/limitador-limitador-8588749977-b5z2x"
Sep 11 11:43:00.038687 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:00.038530    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf9hn\" (UniqueName: \"kubernetes.io/projected/3eb2aba2-8a29-4aa5-b440-3107feab2c7c-kube-api-access-gf9hn\") pod \"limitador-limitador-8588749977-b5z2x\" (UID: \"3eb2aba2-8a29-4aa5-b440-3107feab2c7c\") " pod="kuadrant-system/limitador-limitador-8588749977-b5z2x"
Sep 11 11:43:00.094384 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:00.094358    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-b5z2x"]
Sep 11 11:43:00.125368 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:00.125339    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:43:00.139355 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:00.139327    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf9hn\" (UniqueName: \"kubernetes.io/projected/3eb2aba2-8a29-4aa5-b440-3107feab2c7c-kube-api-access-gf9hn\") pod \"limitador-limitador-8588749977-b5z2x\" (UID: \"3eb2aba2-8a29-4aa5-b440-3107feab2c7c\") " pod="kuadrant-system/limitador-limitador-8588749977-b5z2x"
Sep 11 11:43:00.139517 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:00.139405    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3eb2aba2-8a29-4aa5-b440-3107feab2c7c-config-file\") pod \"limitador-limitador-8588749977-b5z2x\" (UID: \"3eb2aba2-8a29-4aa5-b440-3107feab2c7c\") " pod="kuadrant-system/limitador-limitador-8588749977-b5z2x"
Sep 11 11:43:00.140127 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:00.140104    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3eb2aba2-8a29-4aa5-b440-3107feab2c7c-config-file\") pod \"limitador-limitador-8588749977-b5z2x\" (UID: \"3eb2aba2-8a29-4aa5-b440-3107feab2c7c\") " pod="kuadrant-system/limitador-limitador-8588749977-b5z2x"
Sep 11 11:43:00.154739 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:00.154710    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf9hn\" (UniqueName: \"kubernetes.io/projected/3eb2aba2-8a29-4aa5-b440-3107feab2c7c-kube-api-access-gf9hn\") pod \"limitador-limitador-8588749977-b5z2x\" (UID: \"3eb2aba2-8a29-4aa5-b440-3107feab2c7c\") " pod="kuadrant-system/limitador-limitador-8588749977-b5z2x"
Sep 11 11:43:00.259535 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:00.259503    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:43:00.317375 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:00.317346    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-b5z2x"
Sep 11 11:43:00.599325 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:00.599302    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-b5z2x"]
Sep 11 11:43:00.600971 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:43:00.600946    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eb2aba2_8a29_4aa5_b440_3107feab2c7c.slice/crio-c9af5ca828ef19a512bfccf8476a4745b1da7fcb3cbed94c53c0d71268ca830d WatchSource:0}: Error finding container c9af5ca828ef19a512bfccf8476a4745b1da7fcb3cbed94c53c0d71268ca830d: Status 404 returned error can't find the container with id c9af5ca828ef19a512bfccf8476a4745b1da7fcb3cbed94c53c0d71268ca830d
Sep 11 11:43:00.993496 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:00.993454    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-slcqk"]
Sep 11 11:43:00.998009 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:00.997989    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-slcqk"
Sep 11 11:43:01.005474 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:43:01.005438    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"authorino-7bcfd8f9cd-slcqk\" is forbidden: User \"system:node:ip-10-0-135-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-16.ec2.internal' and this object" podUID="9c878be1-1f9b-42be-a52e-17db4b6dae74" pod="kuadrant-system/authorino-7bcfd8f9cd-slcqk"
Sep 11 11:43:01.006326 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:01.006308    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-qkq5f\""
Sep 11 11:43:01.038139 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:01.038109    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-slcqk"]
Sep 11 11:43:01.046733 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:01.046703    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtr9h\" (UniqueName: \"kubernetes.io/projected/9c878be1-1f9b-42be-a52e-17db4b6dae74-kube-api-access-gtr9h\") pod \"authorino-7bcfd8f9cd-slcqk\" (UID: \"9c878be1-1f9b-42be-a52e-17db4b6dae74\") " pod="kuadrant-system/authorino-7bcfd8f9cd-slcqk"
Sep 11 11:43:01.090706 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:01.090672    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:43:01.147774 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:01.147742    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtr9h\" (UniqueName: \"kubernetes.io/projected/9c878be1-1f9b-42be-a52e-17db4b6dae74-kube-api-access-gtr9h\") pod \"authorino-7bcfd8f9cd-slcqk\" (UID: \"9c878be1-1f9b-42be-a52e-17db4b6dae74\") " pod="kuadrant-system/authorino-7bcfd8f9cd-slcqk"
Sep 11 11:43:01.163434 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:01.163400    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtr9h\" (UniqueName: \"kubernetes.io/projected/9c878be1-1f9b-42be-a52e-17db4b6dae74-kube-api-access-gtr9h\") pod \"authorino-7bcfd8f9cd-slcqk\" (UID: \"9c878be1-1f9b-42be-a52e-17db4b6dae74\") " pod="kuadrant-system/authorino-7bcfd8f9cd-slcqk"
Sep 11 11:43:01.224470 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:01.224439    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:43:01.308368 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:01.308298    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-slcqk"
Sep 11 11:43:01.561494 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:01.561386    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-b5z2x" event={"ID":"3eb2aba2-8a29-4aa5-b440-3107feab2c7c","Type":"ContainerStarted","Data":"c9af5ca828ef19a512bfccf8476a4745b1da7fcb3cbed94c53c0d71268ca830d"}
Sep 11 11:43:01.582099 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:01.582071    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-slcqk"]
Sep 11 11:43:01.584451 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:43:01.584418    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c878be1_1f9b_42be_a52e_17db4b6dae74.slice/crio-927aec737ee50b75eee8f405570c2bdb05c915ef720bdacc91ef4586285dcf65 WatchSource:0}: Error finding container 927aec737ee50b75eee8f405570c2bdb05c915ef720bdacc91ef4586285dcf65: Status 404 returned error can't find the container with id 927aec737ee50b75eee8f405570c2bdb05c915ef720bdacc91ef4586285dcf65
Sep 11 11:43:02.567451 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:02.567411    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-slcqk" event={"ID":"9c878be1-1f9b-42be-a52e-17db4b6dae74","Type":"ContainerStarted","Data":"927aec737ee50b75eee8f405570c2bdb05c915ef720bdacc91ef4586285dcf65"}
Sep 11 11:43:02.572001 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:02.571977    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-b5z2x" event={"ID":"3eb2aba2-8a29-4aa5-b440-3107feab2c7c","Type":"ContainerStarted","Data":"944b8e22780c122dce8846be9c7a014531e2664eebb4be3f6fd2e7140fada47e"}
Sep 11 11:43:02.572121 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:02.572108    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-b5z2x"
Sep 11 11:43:02.605940 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:02.605888    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-b5z2x" podStartSLOduration=2.273014127 podStartE2EDuration="3.605871201s" totalImagesPullingTime="1.332857074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:42:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:43:02.60466306 +0000 UTC m=+523.025269078" watchObservedRunningTime="2026-09-11 11:43:02.605871201 +0000 UTC m=+523.026477283"
Sep 11 11:43:04.794754 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:04.794718    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-slcqk"]
Sep 11 11:43:06.591625 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:06.591588    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-slcqk" event={"ID":"9c878be1-1f9b-42be-a52e-17db4b6dae74","Type":"ContainerStarted","Data":"f9dcd4ca09ccd73336f0134d3d1e11d91fdb8fa812bf15e1591debc41e4ac372"}
Sep 11 11:43:06.592080 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:06.591660    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-slcqk" podUID="9c878be1-1f9b-42be-a52e-17db4b6dae74" containerName="authorino" containerID="cri-o://f9dcd4ca09ccd73336f0134d3d1e11d91fdb8fa812bf15e1591debc41e4ac372" gracePeriod=30
Sep 11 11:43:06.614605 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:06.614554    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-slcqk" podStartSLOduration=2.360725711 podStartE2EDuration="6.614536199s" totalImagesPullingTime="4.253810488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:43:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:43:06.613226496 +0000 UTC m=+527.033832522" watchObservedRunningTime="2026-09-11 11:43:06.614536199 +0000 UTC m=+527.035142226"
Sep 11 11:43:06.831889 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:06.831854    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-slcqk"
Sep 11 11:43:06.892960 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:06.892868    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtr9h\" (UniqueName: \"kubernetes.io/projected/9c878be1-1f9b-42be-a52e-17db4b6dae74-kube-api-access-gtr9h\") pod \"9c878be1-1f9b-42be-a52e-17db4b6dae74\" (UID: \"9c878be1-1f9b-42be-a52e-17db4b6dae74\") "
Sep 11 11:43:06.894999 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:06.894978    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c878be1-1f9b-42be-a52e-17db4b6dae74-kube-api-access-gtr9h" (OuterVolumeSpecName: "kube-api-access-gtr9h") pod "9c878be1-1f9b-42be-a52e-17db4b6dae74" (UID: "9c878be1-1f9b-42be-a52e-17db4b6dae74"). InnerVolumeSpecName "kube-api-access-gtr9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:43:06.994484 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:06.994453    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtr9h\" (UniqueName: \"kubernetes.io/projected/9c878be1-1f9b-42be-a52e-17db4b6dae74-kube-api-access-gtr9h\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:43:07.597715 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:07.597682    2575 generic.go:356] "Generic (PLEG): container finished" podID="9c878be1-1f9b-42be-a52e-17db4b6dae74" containerID="f9dcd4ca09ccd73336f0134d3d1e11d91fdb8fa812bf15e1591debc41e4ac372" exitCode=0
Sep 11 11:43:07.597715 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:07.597720    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-slcqk"
Sep 11 11:43:07.598195 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:07.597740    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-slcqk" event={"ID":"9c878be1-1f9b-42be-a52e-17db4b6dae74","Type":"ContainerDied","Data":"f9dcd4ca09ccd73336f0134d3d1e11d91fdb8fa812bf15e1591debc41e4ac372"}
Sep 11 11:43:07.598195 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:07.597770    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-slcqk" event={"ID":"9c878be1-1f9b-42be-a52e-17db4b6dae74","Type":"ContainerDied","Data":"927aec737ee50b75eee8f405570c2bdb05c915ef720bdacc91ef4586285dcf65"}
Sep 11 11:43:07.598195 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:07.597787    2575 scope.go:117] "RemoveContainer" containerID="f9dcd4ca09ccd73336f0134d3d1e11d91fdb8fa812bf15e1591debc41e4ac372"
Sep 11 11:43:07.607552 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:07.607534    2575 scope.go:117] "RemoveContainer" containerID="f9dcd4ca09ccd73336f0134d3d1e11d91fdb8fa812bf15e1591debc41e4ac372"
Sep 11 11:43:07.607879 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:43:07.607862    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9dcd4ca09ccd73336f0134d3d1e11d91fdb8fa812bf15e1591debc41e4ac372\": container with ID starting with f9dcd4ca09ccd73336f0134d3d1e11d91fdb8fa812bf15e1591debc41e4ac372 not found: ID does not exist" containerID="f9dcd4ca09ccd73336f0134d3d1e11d91fdb8fa812bf15e1591debc41e4ac372"
Sep 11 11:43:07.607927 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:07.607889    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9dcd4ca09ccd73336f0134d3d1e11d91fdb8fa812bf15e1591debc41e4ac372"} err="failed to get container status \"f9dcd4ca09ccd73336f0134d3d1e11d91fdb8fa812bf15e1591debc41e4ac372\": rpc error: code = NotFound desc = could not find container \"f9dcd4ca09ccd73336f0134d3d1e11d91fdb8fa812bf15e1591debc41e4ac372\": container with ID starting with f9dcd4ca09ccd73336f0134d3d1e11d91fdb8fa812bf15e1591debc41e4ac372 not found: ID does not exist"
Sep 11 11:43:07.621440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:07.621414    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-slcqk"]
Sep 11 11:43:07.631800 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:07.628097    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-slcqk"]
Sep 11 11:43:08.081457 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:08.081425    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c878be1-1f9b-42be-a52e-17db4b6dae74" path="/var/lib/kubelet/pods/9c878be1-1f9b-42be-a52e-17db4b6dae74/volumes"
Sep 11 11:43:13.577584 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:13.577504    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-b5z2x"
Sep 11 11:43:13.774215 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:13.774182    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-b5z2x"]
Sep 11 11:43:13.774443 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:13.774398    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-b5z2x" podUID="3eb2aba2-8a29-4aa5-b440-3107feab2c7c" containerName="limitador" containerID="cri-o://944b8e22780c122dce8846be9c7a014531e2664eebb4be3f6fd2e7140fada47e" gracePeriod=30
Sep 11 11:43:14.316213 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:14.316186    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-b5z2x"
Sep 11 11:43:14.361554 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:14.361481    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3eb2aba2-8a29-4aa5-b440-3107feab2c7c-config-file\") pod \"3eb2aba2-8a29-4aa5-b440-3107feab2c7c\" (UID: \"3eb2aba2-8a29-4aa5-b440-3107feab2c7c\") "
Sep 11 11:43:14.361706 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:14.361552    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gf9hn\" (UniqueName: \"kubernetes.io/projected/3eb2aba2-8a29-4aa5-b440-3107feab2c7c-kube-api-access-gf9hn\") pod \"3eb2aba2-8a29-4aa5-b440-3107feab2c7c\" (UID: \"3eb2aba2-8a29-4aa5-b440-3107feab2c7c\") "
Sep 11 11:43:14.361877 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:14.361853    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3eb2aba2-8a29-4aa5-b440-3107feab2c7c-config-file" (OuterVolumeSpecName: "config-file") pod "3eb2aba2-8a29-4aa5-b440-3107feab2c7c" (UID: "3eb2aba2-8a29-4aa5-b440-3107feab2c7c"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:43:14.363845 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:14.363824    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eb2aba2-8a29-4aa5-b440-3107feab2c7c-kube-api-access-gf9hn" (OuterVolumeSpecName: "kube-api-access-gf9hn") pod "3eb2aba2-8a29-4aa5-b440-3107feab2c7c" (UID: "3eb2aba2-8a29-4aa5-b440-3107feab2c7c"). InnerVolumeSpecName "kube-api-access-gf9hn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:43:14.463221 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:14.463193    2575 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3eb2aba2-8a29-4aa5-b440-3107feab2c7c-config-file\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:43:14.463221 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:14.463216    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gf9hn\" (UniqueName: \"kubernetes.io/projected/3eb2aba2-8a29-4aa5-b440-3107feab2c7c-kube-api-access-gf9hn\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:43:14.631828 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:14.631739    2575 generic.go:356] "Generic (PLEG): container finished" podID="3eb2aba2-8a29-4aa5-b440-3107feab2c7c" containerID="944b8e22780c122dce8846be9c7a014531e2664eebb4be3f6fd2e7140fada47e" exitCode=0
Sep 11 11:43:14.631828 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:14.631796    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-b5z2x"
Sep 11 11:43:14.632242 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:14.631843    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-b5z2x" event={"ID":"3eb2aba2-8a29-4aa5-b440-3107feab2c7c","Type":"ContainerDied","Data":"944b8e22780c122dce8846be9c7a014531e2664eebb4be3f6fd2e7140fada47e"}
Sep 11 11:43:14.632242 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:14.631895    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-b5z2x" event={"ID":"3eb2aba2-8a29-4aa5-b440-3107feab2c7c","Type":"ContainerDied","Data":"c9af5ca828ef19a512bfccf8476a4745b1da7fcb3cbed94c53c0d71268ca830d"}
Sep 11 11:43:14.632242 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:14.631920    2575 scope.go:117] "RemoveContainer" containerID="944b8e22780c122dce8846be9c7a014531e2664eebb4be3f6fd2e7140fada47e"
Sep 11 11:43:14.640520 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:14.640503    2575 scope.go:117] "RemoveContainer" containerID="944b8e22780c122dce8846be9c7a014531e2664eebb4be3f6fd2e7140fada47e"
Sep 11 11:43:14.640822 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:43:14.640803    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"944b8e22780c122dce8846be9c7a014531e2664eebb4be3f6fd2e7140fada47e\": container with ID starting with 944b8e22780c122dce8846be9c7a014531e2664eebb4be3f6fd2e7140fada47e not found: ID does not exist" containerID="944b8e22780c122dce8846be9c7a014531e2664eebb4be3f6fd2e7140fada47e"
Sep 11 11:43:14.640877 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:14.640830    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"944b8e22780c122dce8846be9c7a014531e2664eebb4be3f6fd2e7140fada47e"} err="failed to get container status \"944b8e22780c122dce8846be9c7a014531e2664eebb4be3f6fd2e7140fada47e\": rpc error: code = NotFound desc = could not find container \"944b8e22780c122dce8846be9c7a014531e2664eebb4be3f6fd2e7140fada47e\": container with ID starting with 944b8e22780c122dce8846be9c7a014531e2664eebb4be3f6fd2e7140fada47e not found: ID does not exist"
Sep 11 11:43:14.654578 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:14.654556    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-b5z2x"]
Sep 11 11:43:14.659134 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:14.659115    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-b5z2x"]
Sep 11 11:43:16.081288 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:16.081253    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3eb2aba2-8a29-4aa5-b440-3107feab2c7c" path="/var/lib/kubelet/pods/3eb2aba2-8a29-4aa5-b440-3107feab2c7c/volumes"
Sep 11 11:43:18.336570 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:18.336540    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:43:18.465854 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:18.465824    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:43:18.816099 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:18.816062    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-2fh9t"]
Sep 11 11:43:18.816450 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:18.816437    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c878be1-1f9b-42be-a52e-17db4b6dae74" containerName="authorino"
Sep 11 11:43:18.816495 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:18.816453    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c878be1-1f9b-42be-a52e-17db4b6dae74" containerName="authorino"
Sep 11 11:43:18.816495 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:18.816467    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eb2aba2-8a29-4aa5-b440-3107feab2c7c" containerName="limitador"
Sep 11 11:43:18.816495 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:18.816472    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eb2aba2-8a29-4aa5-b440-3107feab2c7c" containerName="limitador"
Sep 11 11:43:18.816588 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:18.816561    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3eb2aba2-8a29-4aa5-b440-3107feab2c7c" containerName="limitador"
Sep 11 11:43:18.816588 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:18.816572    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c878be1-1f9b-42be-a52e-17db4b6dae74" containerName="authorino"
Sep 11 11:43:18.821018 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:18.821000    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2fh9t"
Sep 11 11:43:18.824076 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:43:18.824051    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"postgres-creds\" is forbidden: User \"system:node:ip-10-0-135-16.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-135-16.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" type="*v1.Secret"
Sep 11 11:43:18.824184 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:43:18.824071    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"postgres-69fb4594cf-2fh9t\" is forbidden: User \"system:node:ip-10-0-135-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-135-16.ec2.internal' and this object" podUID="4e9ec7d6-8265-4cae-8661-057ee44fd1cc" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2fh9t"
Sep 11 11:43:18.824184 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:43:18.824090    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-135-16.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-135-16.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Sep 11 11:43:18.824184 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:43:18.824127    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-135-16.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-135-16.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Sep 11 11:43:18.824814 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:43:18.824789    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-bprpg\" is forbidden: User \"system:node:ip-10-0-135-16.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-135-16.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-bprpg\"" type="*v1.Secret"
Sep 11 11:43:18.843174 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:18.843138    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-2fh9t"]
Sep 11 11:43:18.899094 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:18.899060    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbg8j\" (UniqueName: \"kubernetes.io/projected/4e9ec7d6-8265-4cae-8661-057ee44fd1cc-kube-api-access-lbg8j\") pod \"postgres-69fb4594cf-2fh9t\" (UID: \"4e9ec7d6-8265-4cae-8661-057ee44fd1cc\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2fh9t"
Sep 11 11:43:18.899254 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:18.899187    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4e9ec7d6-8265-4cae-8661-057ee44fd1cc-data\") pod \"postgres-69fb4594cf-2fh9t\" (UID: \"4e9ec7d6-8265-4cae-8661-057ee44fd1cc\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2fh9t"
Sep 11 11:43:18.928489 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:18.928464    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:43:18.999697 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:18.999658    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4e9ec7d6-8265-4cae-8661-057ee44fd1cc-data\") pod \"postgres-69fb4594cf-2fh9t\" (UID: \"4e9ec7d6-8265-4cae-8661-057ee44fd1cc\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2fh9t"
Sep 11 11:43:18.999874 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:18.999719    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbg8j\" (UniqueName: \"kubernetes.io/projected/4e9ec7d6-8265-4cae-8661-057ee44fd1cc-kube-api-access-lbg8j\") pod \"postgres-69fb4594cf-2fh9t\" (UID: \"4e9ec7d6-8265-4cae-8661-057ee44fd1cc\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2fh9t"
Sep 11 11:43:19.000032 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:19.000014    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4e9ec7d6-8265-4cae-8661-057ee44fd1cc-data\") pod \"postgres-69fb4594cf-2fh9t\" (UID: \"4e9ec7d6-8265-4cae-8661-057ee44fd1cc\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2fh9t"
Sep 11 11:43:19.937611 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:19.937575    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 11 11:43:20.008874 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:43:20.008828    2575 projected.go:291] Couldn't get configMap odh-ai-gateway-infra/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition
Sep 11 11:43:20.206229 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:20.206143    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 11:43:20.326062 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:20.326033    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 11:43:20.328985 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:43:20.328971    2575 projected.go:196] Error preparing data for projected volume kube-api-access-lbg8j for pod odh-ai-gateway-infra/postgres-69fb4594cf-2fh9t: failed to sync configmap cache: timed out waiting for the condition
Sep 11 11:43:20.329045 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:43:20.329036    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e9ec7d6-8265-4cae-8661-057ee44fd1cc-kube-api-access-lbg8j podName:4e9ec7d6-8265-4cae-8661-057ee44fd1cc nodeName:}" failed. No retries permitted until 2026-09-11 11:43:20.82901241 +0000 UTC m=+541.249618430 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lbg8j" (UniqueName: "kubernetes.io/projected/4e9ec7d6-8265-4cae-8661-057ee44fd1cc-kube-api-access-lbg8j") pod "postgres-69fb4594cf-2fh9t" (UID: "4e9ec7d6-8265-4cae-8661-057ee44fd1cc") : failed to sync configmap cache: timed out waiting for the condition
Sep 11 11:43:20.346985 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:20.346967    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-bprpg\""
Sep 11 11:43:20.917603 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:20.917563    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbg8j\" (UniqueName: \"kubernetes.io/projected/4e9ec7d6-8265-4cae-8661-057ee44fd1cc-kube-api-access-lbg8j\") pod \"postgres-69fb4594cf-2fh9t\" (UID: \"4e9ec7d6-8265-4cae-8661-057ee44fd1cc\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2fh9t"
Sep 11 11:43:20.920236 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:20.920207    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbg8j\" (UniqueName: \"kubernetes.io/projected/4e9ec7d6-8265-4cae-8661-057ee44fd1cc-kube-api-access-lbg8j\") pod \"postgres-69fb4594cf-2fh9t\" (UID: \"4e9ec7d6-8265-4cae-8661-057ee44fd1cc\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-2fh9t"
Sep 11 11:43:20.931227 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:20.931206    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2fh9t"
Sep 11 11:43:21.213808 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:21.213666    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-2fh9t"]
Sep 11 11:43:21.216087 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:43:21.216056    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e9ec7d6_8265_4cae_8661_057ee44fd1cc.slice/crio-6bbb4a0e9429103d692dc9e562160aada3d2544af915b9e1de080fc9f58a6f57 WatchSource:0}: Error finding container 6bbb4a0e9429103d692dc9e562160aada3d2544af915b9e1de080fc9f58a6f57: Status 404 returned error can't find the container with id 6bbb4a0e9429103d692dc9e562160aada3d2544af915b9e1de080fc9f58a6f57
Sep 11 11:43:21.665581 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:21.665545    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2fh9t" event={"ID":"4e9ec7d6-8265-4cae-8661-057ee44fd1cc","Type":"ContainerStarted","Data":"6bbb4a0e9429103d692dc9e562160aada3d2544af915b9e1de080fc9f58a6f57"}
Sep 11 11:43:28.703700 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:28.703660    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2fh9t" event={"ID":"4e9ec7d6-8265-4cae-8661-057ee44fd1cc","Type":"ContainerStarted","Data":"8a286bacbc79f212d97c4f496df9a8f98ff081c5d99c7abbfba20561eb43ce91"}
Sep 11 11:43:28.704179 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:28.703792    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2fh9t"
Sep 11 11:43:28.740113 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:28.740064    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2fh9t" podStartSLOduration=3.880662339 podStartE2EDuration="10.740048492s" totalImagesPullingTime="6.859386153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:43:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:43:28.739435874 +0000 UTC m=+549.160041901" watchObservedRunningTime="2026-09-11 11:43:28.740048492 +0000 UTC m=+549.160654520"
Sep 11 11:43:29.242057 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:29.242027    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:43:34.737600 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:34.737573    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-2fh9t"
Sep 11 11:43:35.170775 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.170679    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:43:35.352258 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.352218    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:43:35.406779 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.406749    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-fdqdw"]
Sep 11 11:43:35.412459 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.412443    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-fdqdw"
Sep 11 11:43:35.416290 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.416268    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 11:43:35.416621 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.416609    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 11:43:35.417149 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.417136    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-qkq5f\""
Sep 11 11:43:35.429576 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.429522    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-fdqdw"]
Sep 11 11:43:35.553020 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.552988    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7ks6\" (UniqueName: \"kubernetes.io/projected/839cede4-e748-4808-a609-e6bcf2ce809a-kube-api-access-g7ks6\") pod \"authorino-557f876f69-fdqdw\" (UID: \"839cede4-e748-4808-a609-e6bcf2ce809a\") " pod="kuadrant-system/authorino-557f876f69-fdqdw"
Sep 11 11:43:35.559925 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.559902    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-fdqdw"]
Sep 11 11:43:35.560179 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:43:35.560160    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-g7ks6], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-fdqdw" podUID="839cede4-e748-4808-a609-e6bcf2ce809a"
Sep 11 11:43:35.606319 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.606294    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-4cnvw"]
Sep 11 11:43:35.610239 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.610221    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-4cnvw"
Sep 11 11:43:35.612775 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.612747    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 11 11:43:35.622577 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.622548    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-4cnvw"]
Sep 11 11:43:35.654469 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.654439    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7ks6\" (UniqueName: \"kubernetes.io/projected/839cede4-e748-4808-a609-e6bcf2ce809a-kube-api-access-g7ks6\") pod \"authorino-557f876f69-fdqdw\" (UID: \"839cede4-e748-4808-a609-e6bcf2ce809a\") " pod="kuadrant-system/authorino-557f876f69-fdqdw"
Sep 11 11:43:35.665298 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.665274    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7ks6\" (UniqueName: \"kubernetes.io/projected/839cede4-e748-4808-a609-e6bcf2ce809a-kube-api-access-g7ks6\") pod \"authorino-557f876f69-fdqdw\" (UID: \"839cede4-e748-4808-a609-e6bcf2ce809a\") " pod="kuadrant-system/authorino-557f876f69-fdqdw"
Sep 11 11:43:35.675323 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.675299    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-4cnvw"]
Sep 11 11:43:35.675608 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:43:35.675586    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-chkjc tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-chkjc tls-cert]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-4cnvw" podUID="afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c"
Sep 11 11:43:35.715436 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.715373    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6c5dc55db6-n645g"]
Sep 11 11:43:35.719247 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.719231    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c5dc55db6-n645g"
Sep 11 11:43:35.734449 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.734429    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-4cnvw"
Sep 11 11:43:35.734549 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.734434    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-fdqdw"
Sep 11 11:43:35.743789 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.743768    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-4cnvw"
Sep 11 11:43:35.744279 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.743788    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c5dc55db6-n645g"]
Sep 11 11:43:35.750902 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.750881    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-fdqdw"
Sep 11 11:43:35.755073 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.755048    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c-tls-cert\") pod \"authorino-6d4dfc649f-4cnvw\" (UID: \"afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c\") " pod="kuadrant-system/authorino-6d4dfc649f-4cnvw"
Sep 11 11:43:35.755176 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.755124    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chkjc\" (UniqueName: \"kubernetes.io/projected/afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c-kube-api-access-chkjc\") pod \"authorino-6d4dfc649f-4cnvw\" (UID: \"afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c\") " pod="kuadrant-system/authorino-6d4dfc649f-4cnvw"
Sep 11 11:43:35.855509 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.855479    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7ks6\" (UniqueName: \"kubernetes.io/projected/839cede4-e748-4808-a609-e6bcf2ce809a-kube-api-access-g7ks6\") pod \"839cede4-e748-4808-a609-e6bcf2ce809a\" (UID: \"839cede4-e748-4808-a609-e6bcf2ce809a\") "
Sep 11 11:43:35.855719 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.855703    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c-tls-cert\") pod \"authorino-6d4dfc649f-4cnvw\" (UID: \"afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c\") " pod="kuadrant-system/authorino-6d4dfc649f-4cnvw"
Sep 11 11:43:35.855784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.855762    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chkjc\" (UniqueName: \"kubernetes.io/projected/afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c-kube-api-access-chkjc\") pod \"authorino-6d4dfc649f-4cnvw\" (UID: \"afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c\") " pod="kuadrant-system/authorino-6d4dfc649f-4cnvw"
Sep 11 11:43:35.855842 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.855792    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6e5f4964-eb11-43af-988d-90114d46ea46-tls-cert\") pod \"authorino-6c5dc55db6-n645g\" (UID: \"6e5f4964-eb11-43af-988d-90114d46ea46\") " pod="kuadrant-system/authorino-6c5dc55db6-n645g"
Sep 11 11:43:35.855842 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.855825    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkpcn\" (UniqueName: \"kubernetes.io/projected/6e5f4964-eb11-43af-988d-90114d46ea46-kube-api-access-kkpcn\") pod \"authorino-6c5dc55db6-n645g\" (UID: \"6e5f4964-eb11-43af-988d-90114d46ea46\") " pod="kuadrant-system/authorino-6c5dc55db6-n645g"
Sep 11 11:43:35.857784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.857756    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/839cede4-e748-4808-a609-e6bcf2ce809a-kube-api-access-g7ks6" (OuterVolumeSpecName: "kube-api-access-g7ks6") pod "839cede4-e748-4808-a609-e6bcf2ce809a" (UID: "839cede4-e748-4808-a609-e6bcf2ce809a"). InnerVolumeSpecName "kube-api-access-g7ks6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:43:35.858168 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.858150    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c-tls-cert\") pod \"authorino-6d4dfc649f-4cnvw\" (UID: \"afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c\") " pod="kuadrant-system/authorino-6d4dfc649f-4cnvw"
Sep 11 11:43:35.868564 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.868540    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chkjc\" (UniqueName: \"kubernetes.io/projected/afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c-kube-api-access-chkjc\") pod \"authorino-6d4dfc649f-4cnvw\" (UID: \"afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c\") " pod="kuadrant-system/authorino-6d4dfc649f-4cnvw"
Sep 11 11:43:35.956917 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.956887    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c-tls-cert\") pod \"afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c\" (UID: \"afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c\") "
Sep 11 11:43:35.957068 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.956933    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chkjc\" (UniqueName: \"kubernetes.io/projected/afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c-kube-api-access-chkjc\") pod \"afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c\" (UID: \"afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c\") "
Sep 11 11:43:35.957108 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.957077    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6e5f4964-eb11-43af-988d-90114d46ea46-tls-cert\") pod \"authorino-6c5dc55db6-n645g\" (UID: \"6e5f4964-eb11-43af-988d-90114d46ea46\") " pod="kuadrant-system/authorino-6c5dc55db6-n645g"
Sep 11 11:43:35.957143 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.957108    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkpcn\" (UniqueName: \"kubernetes.io/projected/6e5f4964-eb11-43af-988d-90114d46ea46-kube-api-access-kkpcn\") pod \"authorino-6c5dc55db6-n645g\" (UID: \"6e5f4964-eb11-43af-988d-90114d46ea46\") " pod="kuadrant-system/authorino-6c5dc55db6-n645g"
Sep 11 11:43:35.957190 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.957179    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7ks6\" (UniqueName: \"kubernetes.io/projected/839cede4-e748-4808-a609-e6bcf2ce809a-kube-api-access-g7ks6\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:43:35.959125 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.959087    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c" (UID: "afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:43:35.959125 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.959104    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c-kube-api-access-chkjc" (OuterVolumeSpecName: "kube-api-access-chkjc") pod "afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c" (UID: "afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c"). InnerVolumeSpecName "kube-api-access-chkjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:43:35.959734 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.959717    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6e5f4964-eb11-43af-988d-90114d46ea46-tls-cert\") pod \"authorino-6c5dc55db6-n645g\" (UID: \"6e5f4964-eb11-43af-988d-90114d46ea46\") " pod="kuadrant-system/authorino-6c5dc55db6-n645g"
Sep 11 11:43:35.970756 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:35.970707    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkpcn\" (UniqueName: \"kubernetes.io/projected/6e5f4964-eb11-43af-988d-90114d46ea46-kube-api-access-kkpcn\") pod \"authorino-6c5dc55db6-n645g\" (UID: \"6e5f4964-eb11-43af-988d-90114d46ea46\") " pod="kuadrant-system/authorino-6c5dc55db6-n645g"
Sep 11 11:43:36.028815 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:36.028786    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c5dc55db6-n645g"
Sep 11 11:43:36.057621 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:36.057592    2575 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c-tls-cert\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:43:36.057621 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:36.057618    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chkjc\" (UniqueName: \"kubernetes.io/projected/afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c-kube-api-access-chkjc\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:43:36.310492 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:36.310467    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c5dc55db6-n645g"]
Sep 11 11:43:36.312612 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:43:36.312588    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e5f4964_eb11_43af_988d_90114d46ea46.slice/crio-7e3dd806627500660b9eb9901ca1520618d61b0051ac944817671da7b7b383f8 WatchSource:0}: Error finding container 7e3dd806627500660b9eb9901ca1520618d61b0051ac944817671da7b7b383f8: Status 404 returned error can't find the container with id 7e3dd806627500660b9eb9901ca1520618d61b0051ac944817671da7b7b383f8
Sep 11 11:43:36.740510 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:36.740481    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-fdqdw"
Sep 11 11:43:36.740510 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:36.740489    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c5dc55db6-n645g" event={"ID":"6e5f4964-eb11-43af-988d-90114d46ea46","Type":"ContainerStarted","Data":"7e3dd806627500660b9eb9901ca1520618d61b0051ac944817671da7b7b383f8"}
Sep 11 11:43:36.740746 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:36.740675    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-4cnvw"
Sep 11 11:43:36.774955 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:36.774923    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-fdqdw"]
Sep 11 11:43:36.777672 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:36.777626    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-fdqdw"]
Sep 11 11:43:36.807084 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:36.807058    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-4cnvw"]
Sep 11 11:43:36.812125 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:36.812101    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-4cnvw"]
Sep 11 11:43:38.080837 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:38.080805    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="839cede4-e748-4808-a609-e6bcf2ce809a" path="/var/lib/kubelet/pods/839cede4-e748-4808-a609-e6bcf2ce809a/volumes"
Sep 11 11:43:38.081400 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:38.081140    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c" path="/var/lib/kubelet/pods/afd1acd2-8bc4-40a8-9c16-6bde5ddaaf2c/volumes"
Sep 11 11:43:39.754795 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:39.754763    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c5dc55db6-n645g" event={"ID":"6e5f4964-eb11-43af-988d-90114d46ea46","Type":"ContainerStarted","Data":"9da4651b97c698a48ef840724b9efbe3fa8bd33a2ecbeecdad86518201cc46fd"}
Sep 11 11:43:39.780661 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:39.780597    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6c5dc55db6-n645g" podStartSLOduration=1.973747528 podStartE2EDuration="4.78058183s" totalImagesPullingTime="2.806834302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:43:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:43:39.779902308 +0000 UTC m=+560.200508337" watchObservedRunningTime="2026-09-11 11:43:39.78058183 +0000 UTC m=+560.201187884"
Sep 11 11:43:44.925918 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:44.925883    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:43:44.968800 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:44.968768    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-58dc578786-kkgdp"]
Sep 11 11:43:44.972550 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:44.972534    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58dc578786-kkgdp"
Sep 11 11:43:44.979151 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:44.978732    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 11:43:44.979151 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:44.978754    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9fdz2\""
Sep 11 11:43:44.979151 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:44.978738    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 11:43:44.979151 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:44.978732    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 11:43:44.987578 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:44.987557    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58dc578786-kkgdp"]
Sep 11 11:43:45.109394 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.109360    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:43:45.135593 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.135569    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16933d55-c845-48e4-b288-6a5e08acbb03-webhook-cert\") pod \"maas-controller-58dc578786-kkgdp\" (UID: \"16933d55-c845-48e4-b288-6a5e08acbb03\") " pod="opendatahub/maas-controller-58dc578786-kkgdp"
Sep 11 11:43:45.135742 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.135609    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16933d55-c845-48e4-b288-6a5e08acbb03-metrics-certs\") pod \"maas-controller-58dc578786-kkgdp\" (UID: \"16933d55-c845-48e4-b288-6a5e08acbb03\") " pod="opendatahub/maas-controller-58dc578786-kkgdp"
Sep 11 11:43:45.135742 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.135701    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfzn7\" (UniqueName: \"kubernetes.io/projected/16933d55-c845-48e4-b288-6a5e08acbb03-kube-api-access-pfzn7\") pod \"maas-controller-58dc578786-kkgdp\" (UID: \"16933d55-c845-48e4-b288-6a5e08acbb03\") " pod="opendatahub/maas-controller-58dc578786-kkgdp"
Sep 11 11:43:45.236889 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.236865    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16933d55-c845-48e4-b288-6a5e08acbb03-webhook-cert\") pod \"maas-controller-58dc578786-kkgdp\" (UID: \"16933d55-c845-48e4-b288-6a5e08acbb03\") " pod="opendatahub/maas-controller-58dc578786-kkgdp"
Sep 11 11:43:45.237017 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.236908    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16933d55-c845-48e4-b288-6a5e08acbb03-metrics-certs\") pod \"maas-controller-58dc578786-kkgdp\" (UID: \"16933d55-c845-48e4-b288-6a5e08acbb03\") " pod="opendatahub/maas-controller-58dc578786-kkgdp"
Sep 11 11:43:45.237017 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.236948    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfzn7\" (UniqueName: \"kubernetes.io/projected/16933d55-c845-48e4-b288-6a5e08acbb03-kube-api-access-pfzn7\") pod \"maas-controller-58dc578786-kkgdp\" (UID: \"16933d55-c845-48e4-b288-6a5e08acbb03\") " pod="opendatahub/maas-controller-58dc578786-kkgdp"
Sep 11 11:43:45.237091 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:43:45.237033    2575 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 11 11:43:45.237126 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:43:45.237108    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16933d55-c845-48e4-b288-6a5e08acbb03-webhook-cert podName:16933d55-c845-48e4-b288-6a5e08acbb03 nodeName:}" failed. No retries permitted until 2026-09-11 11:43:45.737085135 +0000 UTC m=+566.157691140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/16933d55-c845-48e4-b288-6a5e08acbb03-webhook-cert") pod "maas-controller-58dc578786-kkgdp" (UID: "16933d55-c845-48e4-b288-6a5e08acbb03") : secret "maas-controller-webhook-cert" not found
Sep 11 11:43:45.239548 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.239521    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16933d55-c845-48e4-b288-6a5e08acbb03-metrics-certs\") pod \"maas-controller-58dc578786-kkgdp\" (UID: \"16933d55-c845-48e4-b288-6a5e08acbb03\") " pod="opendatahub/maas-controller-58dc578786-kkgdp"
Sep 11 11:43:45.245547 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.245526    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8bb47c4-fsxds"]
Sep 11 11:43:45.249019 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.249002    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8bb47c4-fsxds"
Sep 11 11:43:45.261476 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.261457    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfzn7\" (UniqueName: \"kubernetes.io/projected/16933d55-c845-48e4-b288-6a5e08acbb03-kube-api-access-pfzn7\") pod \"maas-controller-58dc578786-kkgdp\" (UID: \"16933d55-c845-48e4-b288-6a5e08acbb03\") " pod="opendatahub/maas-controller-58dc578786-kkgdp"
Sep 11 11:43:45.280297 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.280262    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8bb47c4-fsxds"]
Sep 11 11:43:45.320372 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.320346    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:43:45.438687 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.438665    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d061500f-6bec-4e87-965d-44533495bf4b-metrics-certs\") pod \"maas-controller-8bb47c4-fsxds\" (UID: \"d061500f-6bec-4e87-965d-44533495bf4b\") " pod="opendatahub/maas-controller-8bb47c4-fsxds"
Sep 11 11:43:45.438810 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.438705    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d061500f-6bec-4e87-965d-44533495bf4b-webhook-cert\") pod \"maas-controller-8bb47c4-fsxds\" (UID: \"d061500f-6bec-4e87-965d-44533495bf4b\") " pod="opendatahub/maas-controller-8bb47c4-fsxds"
Sep 11 11:43:45.438872 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.438854    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfwf4\" (UniqueName: \"kubernetes.io/projected/d061500f-6bec-4e87-965d-44533495bf4b-kube-api-access-wfwf4\") pod \"maas-controller-8bb47c4-fsxds\" (UID: \"d061500f-6bec-4e87-965d-44533495bf4b\") " pod="opendatahub/maas-controller-8bb47c4-fsxds"
Sep 11 11:43:45.535166 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.535087    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:43:45.540163 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.540144    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfwf4\" (UniqueName: \"kubernetes.io/projected/d061500f-6bec-4e87-965d-44533495bf4b-kube-api-access-wfwf4\") pod \"maas-controller-8bb47c4-fsxds\" (UID: \"d061500f-6bec-4e87-965d-44533495bf4b\") " pod="opendatahub/maas-controller-8bb47c4-fsxds"
Sep 11 11:43:45.540244 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.540205    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d061500f-6bec-4e87-965d-44533495bf4b-metrics-certs\") pod \"maas-controller-8bb47c4-fsxds\" (UID: \"d061500f-6bec-4e87-965d-44533495bf4b\") " pod="opendatahub/maas-controller-8bb47c4-fsxds"
Sep 11 11:43:45.540244 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.540239    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d061500f-6bec-4e87-965d-44533495bf4b-webhook-cert\") pod \"maas-controller-8bb47c4-fsxds\" (UID: \"d061500f-6bec-4e87-965d-44533495bf4b\") " pod="opendatahub/maas-controller-8bb47c4-fsxds"
Sep 11 11:43:45.542884 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.542865    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d061500f-6bec-4e87-965d-44533495bf4b-metrics-certs\") pod \"maas-controller-8bb47c4-fsxds\" (UID: \"d061500f-6bec-4e87-965d-44533495bf4b\") " pod="opendatahub/maas-controller-8bb47c4-fsxds"
Sep 11 11:43:45.542981 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.542901    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d061500f-6bec-4e87-965d-44533495bf4b-webhook-cert\") pod \"maas-controller-8bb47c4-fsxds\" (UID: \"d061500f-6bec-4e87-965d-44533495bf4b\") " pod="opendatahub/maas-controller-8bb47c4-fsxds"
Sep 11 11:43:45.550434 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.550411    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfwf4\" (UniqueName: \"kubernetes.io/projected/d061500f-6bec-4e87-965d-44533495bf4b-kube-api-access-wfwf4\") pod \"maas-controller-8bb47c4-fsxds\" (UID: \"d061500f-6bec-4e87-965d-44533495bf4b\") " pod="opendatahub/maas-controller-8bb47c4-fsxds"
Sep 11 11:43:45.559174 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.559155    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8bb47c4-fsxds"
Sep 11 11:43:45.742051 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.742020    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16933d55-c845-48e4-b288-6a5e08acbb03-webhook-cert\") pod \"maas-controller-58dc578786-kkgdp\" (UID: \"16933d55-c845-48e4-b288-6a5e08acbb03\") " pod="opendatahub/maas-controller-58dc578786-kkgdp"
Sep 11 11:43:45.744494 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.744470    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16933d55-c845-48e4-b288-6a5e08acbb03-webhook-cert\") pod \"maas-controller-58dc578786-kkgdp\" (UID: \"16933d55-c845-48e4-b288-6a5e08acbb03\") " pod="opendatahub/maas-controller-58dc578786-kkgdp"
Sep 11 11:43:45.853735 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.853713    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8bb47c4-fsxds"]
Sep 11 11:43:45.856278 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:43:45.856254    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd061500f_6bec_4e87_965d_44533495bf4b.slice/crio-322a0488c9b2c5c2227754e0ef70ff49b31b70b30bd4f167559d7ad649088b4e WatchSource:0}: Error finding container 322a0488c9b2c5c2227754e0ef70ff49b31b70b30bd4f167559d7ad649088b4e: Status 404 returned error can't find the container with id 322a0488c9b2c5c2227754e0ef70ff49b31b70b30bd4f167559d7ad649088b4e
Sep 11 11:43:45.883217 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:45.883202    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58dc578786-kkgdp"
Sep 11 11:43:46.158236 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:46.158217    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58dc578786-kkgdp"]
Sep 11 11:43:46.160242 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:43:46.160214    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16933d55_c845_48e4_b288_6a5e08acbb03.slice/crio-99a58fa75eabb00ea3ae0c51e993dfb4cdbd84fd78b32aff8c3a7627ee68171f WatchSource:0}: Error finding container 99a58fa75eabb00ea3ae0c51e993dfb4cdbd84fd78b32aff8c3a7627ee68171f: Status 404 returned error can't find the container with id 99a58fa75eabb00ea3ae0c51e993dfb4cdbd84fd78b32aff8c3a7627ee68171f
Sep 11 11:43:46.791854 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:46.791815    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8bb47c4-fsxds" event={"ID":"d061500f-6bec-4e87-965d-44533495bf4b","Type":"ContainerStarted","Data":"322a0488c9b2c5c2227754e0ef70ff49b31b70b30bd4f167559d7ad649088b4e"}
Sep 11 11:43:46.793908 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:46.793866    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58dc578786-kkgdp" event={"ID":"16933d55-c845-48e4-b288-6a5e08acbb03","Type":"ContainerStarted","Data":"99a58fa75eabb00ea3ae0c51e993dfb4cdbd84fd78b32aff8c3a7627ee68171f"}
Sep 11 11:43:49.113886 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:49.113753    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:43:49.371172 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:49.371084    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:43:49.812814 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:49.812773    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58dc578786-kkgdp" event={"ID":"16933d55-c845-48e4-b288-6a5e08acbb03","Type":"ContainerStarted","Data":"177122c5bf836220671f7e269719b3756baba61dffb299c009b33d4ce3876c96"}
Sep 11 11:43:49.813024 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:49.812876    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-58dc578786-kkgdp"
Sep 11 11:43:49.814951 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:49.814917    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8bb47c4-fsxds" event={"ID":"d061500f-6bec-4e87-965d-44533495bf4b","Type":"ContainerStarted","Data":"884e1dfe4ba27115df530eb56ce0e6d346b20f1b609c716409dd4aa075f6953d"}
Sep 11 11:43:49.815272 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:49.815172    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8bb47c4-fsxds"
Sep 11 11:43:49.879035 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:49.878987    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-58dc578786-kkgdp" podStartSLOduration=3.268906701 podStartE2EDuration="5.878972979s" totalImagesPullingTime="2.610066278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:43:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:43:49.875655049 +0000 UTC m=+570.296261066" watchObservedRunningTime="2026-09-11 11:43:49.878972979 +0000 UTC m=+570.299579005"
Sep 11 11:43:49.931894 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:43:49.931848    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8bb47c4-fsxds" podStartSLOduration=2.024476678 podStartE2EDuration="4.931831334s" totalImagesPullingTime="2.907354656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:43:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:43:49.930955184 +0000 UTC m=+570.351561212" watchObservedRunningTime="2026-09-11 11:43:49.931831334 +0000 UTC m=+570.352437361"
Sep 11 11:44:00.821310 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:00.821282    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8bb47c4-fsxds"
Sep 11 11:44:00.821692 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:00.821350    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-58dc578786-kkgdp"
Sep 11 11:44:00.923763 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:00.923728    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58dc578786-kkgdp"]
Sep 11 11:44:00.923957 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:00.923918    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-58dc578786-kkgdp" podUID="16933d55-c845-48e4-b288-6a5e08acbb03" containerName="manager" containerID="cri-o://177122c5bf836220671f7e269719b3756baba61dffb299c009b33d4ce3876c96" gracePeriod=30
Sep 11 11:44:01.162388 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:01.162369    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58dc578786-kkgdp"
Sep 11 11:44:01.276331 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:01.276306    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16933d55-c845-48e4-b288-6a5e08acbb03-webhook-cert\") pod \"16933d55-c845-48e4-b288-6a5e08acbb03\" (UID: \"16933d55-c845-48e4-b288-6a5e08acbb03\") "
Sep 11 11:44:01.276524 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:01.276348    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16933d55-c845-48e4-b288-6a5e08acbb03-metrics-certs\") pod \"16933d55-c845-48e4-b288-6a5e08acbb03\" (UID: \"16933d55-c845-48e4-b288-6a5e08acbb03\") "
Sep 11 11:44:01.276524 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:01.276369    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfzn7\" (UniqueName: \"kubernetes.io/projected/16933d55-c845-48e4-b288-6a5e08acbb03-kube-api-access-pfzn7\") pod \"16933d55-c845-48e4-b288-6a5e08acbb03\" (UID: \"16933d55-c845-48e4-b288-6a5e08acbb03\") "
Sep 11 11:44:01.278737 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:01.278702    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16933d55-c845-48e4-b288-6a5e08acbb03-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "16933d55-c845-48e4-b288-6a5e08acbb03" (UID: "16933d55-c845-48e4-b288-6a5e08acbb03"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:44:01.278845 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:01.278753    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16933d55-c845-48e4-b288-6a5e08acbb03-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "16933d55-c845-48e4-b288-6a5e08acbb03" (UID: "16933d55-c845-48e4-b288-6a5e08acbb03"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:44:01.278845 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:01.278758    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16933d55-c845-48e4-b288-6a5e08acbb03-kube-api-access-pfzn7" (OuterVolumeSpecName: "kube-api-access-pfzn7") pod "16933d55-c845-48e4-b288-6a5e08acbb03" (UID: "16933d55-c845-48e4-b288-6a5e08acbb03"). InnerVolumeSpecName "kube-api-access-pfzn7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:44:01.377552 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:01.377494    2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16933d55-c845-48e4-b288-6a5e08acbb03-webhook-cert\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:44:01.377552 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:01.377515    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16933d55-c845-48e4-b288-6a5e08acbb03-metrics-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:44:01.377552 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:01.377525    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfzn7\" (UniqueName: \"kubernetes.io/projected/16933d55-c845-48e4-b288-6a5e08acbb03-kube-api-access-pfzn7\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:44:01.872849 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:01.872819    2575 generic.go:356] "Generic (PLEG): container finished" podID="16933d55-c845-48e4-b288-6a5e08acbb03" containerID="177122c5bf836220671f7e269719b3756baba61dffb299c009b33d4ce3876c96" exitCode=0
Sep 11 11:44:01.873206 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:01.872865    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58dc578786-kkgdp"
Sep 11 11:44:01.873206 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:01.872909    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58dc578786-kkgdp" event={"ID":"16933d55-c845-48e4-b288-6a5e08acbb03","Type":"ContainerDied","Data":"177122c5bf836220671f7e269719b3756baba61dffb299c009b33d4ce3876c96"}
Sep 11 11:44:01.873206 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:01.872955    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58dc578786-kkgdp" event={"ID":"16933d55-c845-48e4-b288-6a5e08acbb03","Type":"ContainerDied","Data":"99a58fa75eabb00ea3ae0c51e993dfb4cdbd84fd78b32aff8c3a7627ee68171f"}
Sep 11 11:44:01.873206 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:01.872978    2575 scope.go:117] "RemoveContainer" containerID="177122c5bf836220671f7e269719b3756baba61dffb299c009b33d4ce3876c96"
Sep 11 11:44:01.882503 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:01.882487    2575 scope.go:117] "RemoveContainer" containerID="177122c5bf836220671f7e269719b3756baba61dffb299c009b33d4ce3876c96"
Sep 11 11:44:01.882781 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:44:01.882764    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"177122c5bf836220671f7e269719b3756baba61dffb299c009b33d4ce3876c96\": container with ID starting with 177122c5bf836220671f7e269719b3756baba61dffb299c009b33d4ce3876c96 not found: ID does not exist" containerID="177122c5bf836220671f7e269719b3756baba61dffb299c009b33d4ce3876c96"
Sep 11 11:44:01.882826 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:01.882792    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"177122c5bf836220671f7e269719b3756baba61dffb299c009b33d4ce3876c96"} err="failed to get container status \"177122c5bf836220671f7e269719b3756baba61dffb299c009b33d4ce3876c96\": rpc error: code = NotFound desc = could not find container \"177122c5bf836220671f7e269719b3756baba61dffb299c009b33d4ce3876c96\": container with ID starting with 177122c5bf836220671f7e269719b3756baba61dffb299c009b33d4ce3876c96 not found: ID does not exist"
Sep 11 11:44:01.898101 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:01.898080    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58dc578786-kkgdp"]
Sep 11 11:44:01.907157 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:01.907136    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-58dc578786-kkgdp"]
Sep 11 11:44:02.081764 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:02.081735    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16933d55-c845-48e4-b288-6a5e08acbb03" path="/var/lib/kubelet/pods/16933d55-c845-48e4-b288-6a5e08acbb03/volumes"
Sep 11 11:44:03.628852 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:03.628823    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:44:03.741944 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:03.741911    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:44:20.640424 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:20.640382    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:44:22.357037 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:22.357004    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:44:22.734402 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:22.734304    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:44:22.794642 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:22.794612    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"]
Sep 11 11:44:22.794994 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:22.794982    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16933d55-c845-48e4-b288-6a5e08acbb03" containerName="manager"
Sep 11 11:44:22.795041 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:22.794996    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="16933d55-c845-48e4-b288-6a5e08acbb03" containerName="manager"
Sep 11 11:44:22.795076 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:22.795070    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="16933d55-c845-48e4-b288-6a5e08acbb03" containerName="manager"
Sep 11 11:44:22.799023 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:22.799007    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"
Sep 11 11:44:22.803822 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:22.803801    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tztb5\""
Sep 11 11:44:22.804655 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:22.804627    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 11:44:22.812443 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:22.812427    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 11 11:44:22.849250 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:22.849221    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"]
Sep 11 11:44:22.985195 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:22.985173    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4060759e-15b8-4435-aa5b-9ffd03644ede-maas-api-tls\") pod \"maas-api-8f6fd9d5d-zbp79\" (UID: \"4060759e-15b8-4435-aa5b-9ffd03644ede\") " pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"
Sep 11 11:44:22.985344 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:22.985237    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4060759e-15b8-4435-aa5b-9ffd03644ede-openshift-service-ca\") pod \"maas-api-8f6fd9d5d-zbp79\" (UID: \"4060759e-15b8-4435-aa5b-9ffd03644ede\") " pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"
Sep 11 11:44:22.985344 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:22.985272    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45p6h\" (UniqueName: \"kubernetes.io/projected/4060759e-15b8-4435-aa5b-9ffd03644ede-kube-api-access-45p6h\") pod \"maas-api-8f6fd9d5d-zbp79\" (UID: \"4060759e-15b8-4435-aa5b-9ffd03644ede\") " pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"
Sep 11 11:44:22.985344 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:22.985289    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4060759e-15b8-4435-aa5b-9ffd03644ede-metrics-certs\") pod \"maas-api-8f6fd9d5d-zbp79\" (UID: \"4060759e-15b8-4435-aa5b-9ffd03644ede\") " pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"
Sep 11 11:44:23.065463 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:23.065430    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:44:23.086046 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:23.086022    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4060759e-15b8-4435-aa5b-9ffd03644ede-maas-api-tls\") pod \"maas-api-8f6fd9d5d-zbp79\" (UID: \"4060759e-15b8-4435-aa5b-9ffd03644ede\") " pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"
Sep 11 11:44:23.086120 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:23.086085    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4060759e-15b8-4435-aa5b-9ffd03644ede-openshift-service-ca\") pod \"maas-api-8f6fd9d5d-zbp79\" (UID: \"4060759e-15b8-4435-aa5b-9ffd03644ede\") " pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"
Sep 11 11:44:23.086120 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:23.086110    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45p6h\" (UniqueName: \"kubernetes.io/projected/4060759e-15b8-4435-aa5b-9ffd03644ede-kube-api-access-45p6h\") pod \"maas-api-8f6fd9d5d-zbp79\" (UID: \"4060759e-15b8-4435-aa5b-9ffd03644ede\") " pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"
Sep 11 11:44:23.086193 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:23.086133    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4060759e-15b8-4435-aa5b-9ffd03644ede-metrics-certs\") pod \"maas-api-8f6fd9d5d-zbp79\" (UID: \"4060759e-15b8-4435-aa5b-9ffd03644ede\") " pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"
Sep 11 11:44:23.086775 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:23.086740    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4060759e-15b8-4435-aa5b-9ffd03644ede-openshift-service-ca\") pod \"maas-api-8f6fd9d5d-zbp79\" (UID: \"4060759e-15b8-4435-aa5b-9ffd03644ede\") " pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"
Sep 11 11:44:23.088675 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:23.088658    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4060759e-15b8-4435-aa5b-9ffd03644ede-maas-api-tls\") pod \"maas-api-8f6fd9d5d-zbp79\" (UID: \"4060759e-15b8-4435-aa5b-9ffd03644ede\") " pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"
Sep 11 11:44:23.088754 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:23.088708    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4060759e-15b8-4435-aa5b-9ffd03644ede-metrics-certs\") pod \"maas-api-8f6fd9d5d-zbp79\" (UID: \"4060759e-15b8-4435-aa5b-9ffd03644ede\") " pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"
Sep 11 11:44:23.106332 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:23.106311    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45p6h\" (UniqueName: \"kubernetes.io/projected/4060759e-15b8-4435-aa5b-9ffd03644ede-kube-api-access-45p6h\") pod \"maas-api-8f6fd9d5d-zbp79\" (UID: \"4060759e-15b8-4435-aa5b-9ffd03644ede\") " pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"
Sep 11 11:44:23.109245 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:23.109226    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"
Sep 11 11:44:23.412757 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:23.412683    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"]
Sep 11 11:44:23.420751 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:44:23.420728    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4060759e_15b8_4435_aa5b_9ffd03644ede.slice/crio-79120349297d8105b2369b606b40f096dc51b235d4c6f0b935f48aa34c311f11 WatchSource:0}: Error finding container 79120349297d8105b2369b606b40f096dc51b235d4c6f0b935f48aa34c311f11: Status 404 returned error can't find the container with id 79120349297d8105b2369b606b40f096dc51b235d4c6f0b935f48aa34c311f11
Sep 11 11:44:23.422162 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:23.422146    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 11:44:23.978453 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:23.978416    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79" event={"ID":"4060759e-15b8-4435-aa5b-9ffd03644ede","Type":"ContainerStarted","Data":"79120349297d8105b2369b606b40f096dc51b235d4c6f0b935f48aa34c311f11"}
Sep 11 11:44:24.111252 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:24.111219    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:44:24.378150 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:24.378120    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:44:24.641860 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:24.641778    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-qqpzb"]
Sep 11 11:44:24.645524 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:24.645501    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-qqpzb"
Sep 11 11:44:24.648946 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:24.648898    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 11 11:44:24.649283 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:24.649261    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-l2rbs\""
Sep 11 11:44:24.686066 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:24.686036    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-qqpzb"]
Sep 11 11:44:24.803585 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:24.803552    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66d1dfc4-81b5-49fc-bfa7-5c023917525a-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-qqpzb\" (UID: \"66d1dfc4-81b5-49fc-bfa7-5c023917525a\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qqpzb"
Sep 11 11:44:24.803797 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:24.803690    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4s87\" (UniqueName: \"kubernetes.io/projected/66d1dfc4-81b5-49fc-bfa7-5c023917525a-kube-api-access-q4s87\") pod \"payload-processing-5b54b7fbb7-qqpzb\" (UID: \"66d1dfc4-81b5-49fc-bfa7-5c023917525a\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qqpzb"
Sep 11 11:44:24.904362 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:24.904279    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4s87\" (UniqueName: \"kubernetes.io/projected/66d1dfc4-81b5-49fc-bfa7-5c023917525a-kube-api-access-q4s87\") pod \"payload-processing-5b54b7fbb7-qqpzb\" (UID: \"66d1dfc4-81b5-49fc-bfa7-5c023917525a\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qqpzb"
Sep 11 11:44:24.904530 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:24.904411    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66d1dfc4-81b5-49fc-bfa7-5c023917525a-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-qqpzb\" (UID: \"66d1dfc4-81b5-49fc-bfa7-5c023917525a\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qqpzb"
Sep 11 11:44:24.905264 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:24.905233    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66d1dfc4-81b5-49fc-bfa7-5c023917525a-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-qqpzb\" (UID: \"66d1dfc4-81b5-49fc-bfa7-5c023917525a\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qqpzb"
Sep 11 11:44:24.930526 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:24.930499    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4s87\" (UniqueName: \"kubernetes.io/projected/66d1dfc4-81b5-49fc-bfa7-5c023917525a-kube-api-access-q4s87\") pod \"payload-processing-5b54b7fbb7-qqpzb\" (UID: \"66d1dfc4-81b5-49fc-bfa7-5c023917525a\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qqpzb"
Sep 11 11:44:24.957210 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:24.957178    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-qqpzb"
Sep 11 11:44:25.390371 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:25.390349    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-qqpzb"]
Sep 11 11:44:25.393165 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:44:25.393135    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66d1dfc4_81b5_49fc_bfa7_5c023917525a.slice/crio-80f3aa7e20eddaaf1d20d44e12831e2595551645a48da296716a4457ca73e22a WatchSource:0}: Error finding container 80f3aa7e20eddaaf1d20d44e12831e2595551645a48da296716a4457ca73e22a: Status 404 returned error can't find the container with id 80f3aa7e20eddaaf1d20d44e12831e2595551645a48da296716a4457ca73e22a
Sep 11 11:44:25.993202 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:25.993160    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79" event={"ID":"4060759e-15b8-4435-aa5b-9ffd03644ede","Type":"ContainerStarted","Data":"e73d47aacef629f907185253673fb8174ab1238500883e4a2d2bd98f812386c2"}
Sep 11 11:44:25.993774 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:25.993248    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"
Sep 11 11:44:25.994977 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:25.994946    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-qqpzb" event={"ID":"66d1dfc4-81b5-49fc-bfa7-5c023917525a","Type":"ContainerStarted","Data":"80f3aa7e20eddaaf1d20d44e12831e2595551645a48da296716a4457ca73e22a"}
Sep 11 11:44:26.052296 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:26.052232    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79" podStartSLOduration=2.356379178 podStartE2EDuration="4.052210563s" totalImagesPullingTime="1.695831385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:44:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:44:26.050178845 +0000 UTC m=+606.470784873" watchObservedRunningTime="2026-09-11 11:44:26.052210563 +0000 UTC m=+606.472816592"
Sep 11 11:44:27.001493 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:27.001460    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-qqpzb" event={"ID":"66d1dfc4-81b5-49fc-bfa7-5c023917525a","Type":"ContainerStarted","Data":"0af3b58399d452834d610f2936d856425df56e9f7630d262d7df37a3468bcbf4"}
Sep 11 11:44:27.039073 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:27.039032    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-qqpzb" podStartSLOduration=1.57185926 podStartE2EDuration="3.039018379s" totalImagesPullingTime="1.467159119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:44:27.037905533 +0000 UTC m=+607.458511560" watchObservedRunningTime="2026-09-11 11:44:27.039018379 +0000 UTC m=+607.459624407"
Sep 11 11:44:28.006692 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:28.006659    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-qqpzb"
Sep 11 11:44:32.007184 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:32.007159    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"
Sep 11 11:44:33.204201 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:33.204163    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:44:33.386297 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:33.386268    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:44:39.013318 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:39.013288    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-qqpzb"
Sep 11 11:44:50.694206 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.694130    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"]
Sep 11 11:44:50.697408 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.697388    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:44:50.703715 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.703692    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 11:44:50.704991 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.704967    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 11 11:44:50.704991 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.704990    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 11:44:50.705181 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.705056    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7cwhz\""
Sep 11 11:44:50.726606 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.726584    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"]
Sep 11 11:44:50.798571 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.798538    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:44:50.829467 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.829447    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c023d59-bd7e-4d8a-884e-577d5cfb37c0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b\" (UID: \"3c023d59-bd7e-4d8a-884e-577d5cfb37c0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:44:50.829588 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.829504    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c023d59-bd7e-4d8a-884e-577d5cfb37c0-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b\" (UID: \"3c023d59-bd7e-4d8a-884e-577d5cfb37c0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:44:50.829588 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.829522    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c023d59-bd7e-4d8a-884e-577d5cfb37c0-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b\" (UID: \"3c023d59-bd7e-4d8a-884e-577d5cfb37c0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:44:50.829588 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.829575    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c023d59-bd7e-4d8a-884e-577d5cfb37c0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b\" (UID: \"3c023d59-bd7e-4d8a-884e-577d5cfb37c0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:44:50.829778 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.829625    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l5bt\" (UniqueName: \"kubernetes.io/projected/3c023d59-bd7e-4d8a-884e-577d5cfb37c0-kube-api-access-4l5bt\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b\" (UID: \"3c023d59-bd7e-4d8a-884e-577d5cfb37c0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:44:50.829833 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.829796    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c023d59-bd7e-4d8a-884e-577d5cfb37c0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b\" (UID: \"3c023d59-bd7e-4d8a-884e-577d5cfb37c0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:44:50.930695 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.930660    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c023d59-bd7e-4d8a-884e-577d5cfb37c0-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b\" (UID: \"3c023d59-bd7e-4d8a-884e-577d5cfb37c0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:44:50.930695 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.930697    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c023d59-bd7e-4d8a-884e-577d5cfb37c0-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b\" (UID: \"3c023d59-bd7e-4d8a-884e-577d5cfb37c0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:44:50.930928 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.930717    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c023d59-bd7e-4d8a-884e-577d5cfb37c0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b\" (UID: \"3c023d59-bd7e-4d8a-884e-577d5cfb37c0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:44:50.930928 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.930745    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l5bt\" (UniqueName: \"kubernetes.io/projected/3c023d59-bd7e-4d8a-884e-577d5cfb37c0-kube-api-access-4l5bt\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b\" (UID: \"3c023d59-bd7e-4d8a-884e-577d5cfb37c0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:44:50.930928 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.930851    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c023d59-bd7e-4d8a-884e-577d5cfb37c0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b\" (UID: \"3c023d59-bd7e-4d8a-884e-577d5cfb37c0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:44:50.930928 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.930874    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c023d59-bd7e-4d8a-884e-577d5cfb37c0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b\" (UID: \"3c023d59-bd7e-4d8a-884e-577d5cfb37c0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:44:50.931211 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.931187    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c023d59-bd7e-4d8a-884e-577d5cfb37c0-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b\" (UID: \"3c023d59-bd7e-4d8a-884e-577d5cfb37c0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:44:50.931283 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.931209    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c023d59-bd7e-4d8a-884e-577d5cfb37c0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b\" (UID: \"3c023d59-bd7e-4d8a-884e-577d5cfb37c0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:44:50.931283 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.931235    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c023d59-bd7e-4d8a-884e-577d5cfb37c0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b\" (UID: \"3c023d59-bd7e-4d8a-884e-577d5cfb37c0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:44:50.933134 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.933112    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c023d59-bd7e-4d8a-884e-577d5cfb37c0-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b\" (UID: \"3c023d59-bd7e-4d8a-884e-577d5cfb37c0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:44:50.933301 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.933284    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c023d59-bd7e-4d8a-884e-577d5cfb37c0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b\" (UID: \"3c023d59-bd7e-4d8a-884e-577d5cfb37c0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:44:50.948845 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:50.948788    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l5bt\" (UniqueName: \"kubernetes.io/projected/3c023d59-bd7e-4d8a-884e-577d5cfb37c0-kube-api-access-4l5bt\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b\" (UID: \"3c023d59-bd7e-4d8a-884e-577d5cfb37c0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:44:51.008858 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:51.008827    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:44:51.308003 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:51.307944    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"]
Sep 11 11:44:51.309243 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:44:51.309213    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c023d59_bd7e_4d8a_884e_577d5cfb37c0.slice/crio-c7a21c41407ffacecced112ad44a7a1bd089ed903718e40109f3d799b7ce2e53 WatchSource:0}: Error finding container c7a21c41407ffacecced112ad44a7a1bd089ed903718e40109f3d799b7ce2e53: Status 404 returned error can't find the container with id c7a21c41407ffacecced112ad44a7a1bd089ed903718e40109f3d799b7ce2e53
Sep 11 11:44:52.123826 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:52.123791    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b" event={"ID":"3c023d59-bd7e-4d8a-884e-577d5cfb37c0","Type":"ContainerStarted","Data":"c7a21c41407ffacecced112ad44a7a1bd089ed903718e40109f3d799b7ce2e53"}
Sep 11 11:44:52.920833 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:52.920799    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:44:53.129873 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:53.129835    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b" event={"ID":"3c023d59-bd7e-4d8a-884e-577d5cfb37c0","Type":"ContainerStarted","Data":"0a1c89eee3efd154117658c4daefe38a6d40542f53e66bb16f2b8f3c76a933a5"}
Sep 11 11:44:53.159810 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:44:53.159765    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b" podStartSLOduration=1.992314685 podStartE2EDuration="3.159750155s" totalImagesPullingTime="1.16743547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:44:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:44:53.156793776 +0000 UTC m=+633.577399803" watchObservedRunningTime="2026-09-11 11:44:53.159750155 +0000 UTC m=+633.580356160"
Sep 11 11:45:01.009173 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:45:01.009136    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:45:01.009617 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:45:01.009206    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:45:01.021896 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:45:01.021870    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:45:01.180366 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:45:01.180341    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b"
Sep 11 11:45:03.283536 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:45:03.283504    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:45:19.679880 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:45:19.679846    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:45:30.713911 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:45:30.713874    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:45:34.885226 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:45:34.885188    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:45:46.672539 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:45:46.672496    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:45:47.080798 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:45:47.080765    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:45:59.402673 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:45:59.402624    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:46:09.615951 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:46:09.615915    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:46:48.519129 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:46:48.519092    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:14.533152 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:14.533119    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8b66479c9-6zb8s"]
Sep 11 11:47:14.536825 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:14.536807    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b66479c9-6zb8s"
Sep 11 11:47:14.555153 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:14.555088    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b66479c9-6zb8s"]
Sep 11 11:47:14.664788 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:14.664754    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wznkv\" (UniqueName: \"kubernetes.io/projected/c879f287-a93c-4ac0-9546-62a3e0405308-kube-api-access-wznkv\") pod \"authorino-8b66479c9-6zb8s\" (UID: \"c879f287-a93c-4ac0-9546-62a3e0405308\") " pod="kuadrant-system/authorino-8b66479c9-6zb8s"
Sep 11 11:47:14.664948 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:14.664813    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c879f287-a93c-4ac0-9546-62a3e0405308-tls-cert\") pod \"authorino-8b66479c9-6zb8s\" (UID: \"c879f287-a93c-4ac0-9546-62a3e0405308\") " pod="kuadrant-system/authorino-8b66479c9-6zb8s"
Sep 11 11:47:14.765667 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:14.765618    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wznkv\" (UniqueName: \"kubernetes.io/projected/c879f287-a93c-4ac0-9546-62a3e0405308-kube-api-access-wznkv\") pod \"authorino-8b66479c9-6zb8s\" (UID: \"c879f287-a93c-4ac0-9546-62a3e0405308\") " pod="kuadrant-system/authorino-8b66479c9-6zb8s"
Sep 11 11:47:14.765838 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:14.765711    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c879f287-a93c-4ac0-9546-62a3e0405308-tls-cert\") pod \"authorino-8b66479c9-6zb8s\" (UID: \"c879f287-a93c-4ac0-9546-62a3e0405308\") " pod="kuadrant-system/authorino-8b66479c9-6zb8s"
Sep 11 11:47:14.768179 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:14.768160    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c879f287-a93c-4ac0-9546-62a3e0405308-tls-cert\") pod \"authorino-8b66479c9-6zb8s\" (UID: \"c879f287-a93c-4ac0-9546-62a3e0405308\") " pod="kuadrant-system/authorino-8b66479c9-6zb8s"
Sep 11 11:47:14.774483 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:14.774461    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wznkv\" (UniqueName: \"kubernetes.io/projected/c879f287-a93c-4ac0-9546-62a3e0405308-kube-api-access-wznkv\") pod \"authorino-8b66479c9-6zb8s\" (UID: \"c879f287-a93c-4ac0-9546-62a3e0405308\") " pod="kuadrant-system/authorino-8b66479c9-6zb8s"
Sep 11 11:47:14.851795 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:14.851716    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b66479c9-6zb8s"
Sep 11 11:47:15.140432 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:15.140359    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b66479c9-6zb8s"]
Sep 11 11:47:15.143831 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:47:15.143802    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc879f287_a93c_4ac0_9546_62a3e0405308.slice/crio-92143c2c8ae3dc9ca864c2485d0831ea2fe9d738a48582d68f982d2973df2143 WatchSource:0}: Error finding container 92143c2c8ae3dc9ca864c2485d0831ea2fe9d738a48582d68f982d2973df2143: Status 404 returned error can't find the container with id 92143c2c8ae3dc9ca864c2485d0831ea2fe9d738a48582d68f982d2973df2143
Sep 11 11:47:15.837905 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:15.837877    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b66479c9-6zb8s" event={"ID":"c879f287-a93c-4ac0-9546-62a3e0405308","Type":"ContainerStarted","Data":"92143c2c8ae3dc9ca864c2485d0831ea2fe9d738a48582d68f982d2973df2143"}
Sep 11 11:47:16.845808 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:16.845762    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b66479c9-6zb8s" event={"ID":"c879f287-a93c-4ac0-9546-62a3e0405308","Type":"ContainerStarted","Data":"b6d3d24b3bb55edf402ed3ba68522ad0a28b73ac6f13f9a60b3e795c7783627c"}
Sep 11 11:47:16.875579 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:16.875497    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-8b66479c9-6zb8s" podStartSLOduration=2.193314035 podStartE2EDuration="2.875470544s" totalImagesPullingTime="682.156509ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:47:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:47:16.87261127 +0000 UTC m=+777.293217297" watchObservedRunningTime="2026-09-11 11:47:16.875470544 +0000 UTC m=+777.296076575"
Sep 11 11:47:16.922016 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:16.921985    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c5dc55db6-n645g"]
Sep 11 11:47:16.922177 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:16.922153    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6c5dc55db6-n645g" podUID="6e5f4964-eb11-43af-988d-90114d46ea46" containerName="authorino" containerID="cri-o://9da4651b97c698a48ef840724b9efbe3fa8bd33a2ecbeecdad86518201cc46fd" gracePeriod=30
Sep 11 11:47:17.198981 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:17.198803    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c5dc55db6-n645g"
Sep 11 11:47:17.292678 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:17.292604    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6e5f4964-eb11-43af-988d-90114d46ea46-tls-cert\") pod \"6e5f4964-eb11-43af-988d-90114d46ea46\" (UID: \"6e5f4964-eb11-43af-988d-90114d46ea46\") "
Sep 11 11:47:17.292851 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:17.292700    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkpcn\" (UniqueName: \"kubernetes.io/projected/6e5f4964-eb11-43af-988d-90114d46ea46-kube-api-access-kkpcn\") pod \"6e5f4964-eb11-43af-988d-90114d46ea46\" (UID: \"6e5f4964-eb11-43af-988d-90114d46ea46\") "
Sep 11 11:47:17.294819 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:17.294795    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e5f4964-eb11-43af-988d-90114d46ea46-kube-api-access-kkpcn" (OuterVolumeSpecName: "kube-api-access-kkpcn") pod "6e5f4964-eb11-43af-988d-90114d46ea46" (UID: "6e5f4964-eb11-43af-988d-90114d46ea46"). InnerVolumeSpecName "kube-api-access-kkpcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:47:17.305295 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:17.305271    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e5f4964-eb11-43af-988d-90114d46ea46-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "6e5f4964-eb11-43af-988d-90114d46ea46" (UID: "6e5f4964-eb11-43af-988d-90114d46ea46"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:47:17.394443 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:17.394359    2575 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6e5f4964-eb11-43af-988d-90114d46ea46-tls-cert\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:47:17.394443 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:17.394390    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkpcn\" (UniqueName: \"kubernetes.io/projected/6e5f4964-eb11-43af-988d-90114d46ea46-kube-api-access-kkpcn\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:47:17.567065 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:17.566980    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:17.692378 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:17.692287    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:17.853913 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:17.853879    2575 generic.go:356] "Generic (PLEG): container finished" podID="6e5f4964-eb11-43af-988d-90114d46ea46" containerID="9da4651b97c698a48ef840724b9efbe3fa8bd33a2ecbeecdad86518201cc46fd" exitCode=0
Sep 11 11:47:17.854398 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:17.853920    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c5dc55db6-n645g" event={"ID":"6e5f4964-eb11-43af-988d-90114d46ea46","Type":"ContainerDied","Data":"9da4651b97c698a48ef840724b9efbe3fa8bd33a2ecbeecdad86518201cc46fd"}
Sep 11 11:47:17.854398 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:17.853937    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c5dc55db6-n645g"
Sep 11 11:47:17.854398 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:17.853974    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c5dc55db6-n645g" event={"ID":"6e5f4964-eb11-43af-988d-90114d46ea46","Type":"ContainerDied","Data":"7e3dd806627500660b9eb9901ca1520618d61b0051ac944817671da7b7b383f8"}
Sep 11 11:47:17.854398 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:17.853998    2575 scope.go:117] "RemoveContainer" containerID="9da4651b97c698a48ef840724b9efbe3fa8bd33a2ecbeecdad86518201cc46fd"
Sep 11 11:47:17.863869 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:17.863850    2575 scope.go:117] "RemoveContainer" containerID="9da4651b97c698a48ef840724b9efbe3fa8bd33a2ecbeecdad86518201cc46fd"
Sep 11 11:47:17.864216 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:47:17.864192    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9da4651b97c698a48ef840724b9efbe3fa8bd33a2ecbeecdad86518201cc46fd\": container with ID starting with 9da4651b97c698a48ef840724b9efbe3fa8bd33a2ecbeecdad86518201cc46fd not found: ID does not exist" containerID="9da4651b97c698a48ef840724b9efbe3fa8bd33a2ecbeecdad86518201cc46fd"
Sep 11 11:47:17.864285 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:17.864252    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9da4651b97c698a48ef840724b9efbe3fa8bd33a2ecbeecdad86518201cc46fd"} err="failed to get container status \"9da4651b97c698a48ef840724b9efbe3fa8bd33a2ecbeecdad86518201cc46fd\": rpc error: code = NotFound desc = could not find container \"9da4651b97c698a48ef840724b9efbe3fa8bd33a2ecbeecdad86518201cc46fd\": container with ID starting with 9da4651b97c698a48ef840724b9efbe3fa8bd33a2ecbeecdad86518201cc46fd not found: ID does not exist"
Sep 11 11:47:17.894753 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:17.894700    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c5dc55db6-n645g"]
Sep 11 11:47:17.898708 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:17.898682    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6c5dc55db6-n645g"]
Sep 11 11:47:18.082161 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:18.082120    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e5f4964-eb11-43af-988d-90114d46ea46" path="/var/lib/kubelet/pods/6e5f4964-eb11-43af-988d-90114d46ea46/volumes"
Sep 11 11:47:18.679460 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:18.679421    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:18.778263 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:18.778228    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:32.751380 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:32.751342    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:32.815158 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:32.815125    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:33.871841 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:33.871808    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:34.009217 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:34.009177    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:34.301158 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:34.301120    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:34.400843 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:34.400810    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:35.318614 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:35.318576    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:35.458652 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:35.458610    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:35.759797 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:35.759770    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:35.922437 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:35.922406    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8023-4f16ea"]
Sep 11 11:47:35.922811 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:35.922799    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e5f4964-eb11-43af-988d-90114d46ea46" containerName="authorino"
Sep 11 11:47:35.922861 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:35.922815    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e5f4964-eb11-43af-988d-90114d46ea46" containerName="authorino"
Sep 11 11:47:35.922897 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:35.922891    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e5f4964-eb11-43af-988d-90114d46ea46" containerName="authorino"
Sep 11 11:47:35.927384 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:35.927368    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8023-4f16ea"
Sep 11 11:47:35.930330 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:35.930314    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-5xz2c\""
Sep 11 11:47:35.934061 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:35.934032    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8023-4f16ea"]
Sep 11 11:47:35.950675 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:35.950650    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8zgw\" (UniqueName: \"kubernetes.io/projected/fb51ff15-a116-49c3-b7f6-2fea3ae4caea-kube-api-access-d8zgw\") pod \"test-curl-8023-4f16ea\" (UID: \"fb51ff15-a116-49c3-b7f6-2fea3ae4caea\") " pod="opendatahub/test-curl-8023-4f16ea"
Sep 11 11:47:36.052051 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:36.051964    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8zgw\" (UniqueName: \"kubernetes.io/projected/fb51ff15-a116-49c3-b7f6-2fea3ae4caea-kube-api-access-d8zgw\") pod \"test-curl-8023-4f16ea\" (UID: \"fb51ff15-a116-49c3-b7f6-2fea3ae4caea\") " pod="opendatahub/test-curl-8023-4f16ea"
Sep 11 11:47:36.062593 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:36.062571    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8zgw\" (UniqueName: \"kubernetes.io/projected/fb51ff15-a116-49c3-b7f6-2fea3ae4caea-kube-api-access-d8zgw\") pod \"test-curl-8023-4f16ea\" (UID: \"fb51ff15-a116-49c3-b7f6-2fea3ae4caea\") " pod="opendatahub/test-curl-8023-4f16ea"
Sep 11 11:47:36.237808 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:36.237773    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8023-4f16ea"
Sep 11 11:47:36.508081 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:36.508060    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8023-4f16ea"]
Sep 11 11:47:36.509940 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:47:36.509916    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb51ff15_a116_49c3_b7f6_2fea3ae4caea.slice/crio-cfd50c8ed7857d08cddc7bfaa2ff7a1c42ef28d388d3009ffa7d20395b3013a3 WatchSource:0}: Error finding container cfd50c8ed7857d08cddc7bfaa2ff7a1c42ef28d388d3009ffa7d20395b3013a3: Status 404 returned error can't find the container with id cfd50c8ed7857d08cddc7bfaa2ff7a1c42ef28d388d3009ffa7d20395b3013a3
Sep 11 11:47:36.951008 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:36.950928    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8023-4f16ea" event={"ID":"fb51ff15-a116-49c3-b7f6-2fea3ae4caea","Type":"ContainerStarted","Data":"cfd50c8ed7857d08cddc7bfaa2ff7a1c42ef28d388d3009ffa7d20395b3013a3"}
Sep 11 11:47:37.264278 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:37.264244    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:37.377044 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:37.377011    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:38.481826 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:38.480846    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"]
Sep 11 11:47:38.490552 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:38.490526    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"
Sep 11 11:47:38.493424 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:38.493401    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-dbe0e7-serving-cert\""
Sep 11 11:47:38.502514 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:38.502488    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"]
Sep 11 11:47:38.566607 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:38.566569    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:38.586880 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:38.586850    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp\" (UID: \"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"
Sep 11 11:47:38.587050 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:38.586886    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qllbk\" (UniqueName: \"kubernetes.io/projected/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-kube-api-access-qllbk\") pod \"maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp\" (UID: \"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"
Sep 11 11:47:38.587117 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:38.587052    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-metrics-certs\") pod \"maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp\" (UID: \"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"
Sep 11 11:47:38.587117 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:38.587091    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-maas-api-tls\") pod \"maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp\" (UID: \"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"
Sep 11 11:47:38.665521 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:38.665491    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:38.687787 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:38.687753    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-metrics-certs\") pod \"maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp\" (UID: \"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"
Sep 11 11:47:38.687787 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:38.687801    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-maas-api-tls\") pod \"maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp\" (UID: \"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"
Sep 11 11:47:38.688035 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:38.687887    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp\" (UID: \"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"
Sep 11 11:47:38.688035 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:47:38.687984    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-serving-cert: secret "maas-api-e2e-inf-a-dbe0e7-serving-cert" not found
Sep 11 11:47:38.688139 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:38.688052    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qllbk\" (UniqueName: \"kubernetes.io/projected/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-kube-api-access-qllbk\") pod \"maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp\" (UID: \"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"
Sep 11 11:47:38.688139 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:47:38.688063    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-maas-api-tls podName:7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a nodeName:}" failed. No retries permitted until 2026-09-11 11:47:39.188039399 +0000 UTC m=+799.608645406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-maas-api-tls") pod "maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp" (UID: "7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a") : secret "maas-api-e2e-inf-a-dbe0e7-serving-cert" not found
Sep 11 11:47:38.688679 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:38.688657    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp\" (UID: \"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"
Sep 11 11:47:38.690731 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:38.690712    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-metrics-certs\") pod \"maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp\" (UID: \"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"
Sep 11 11:47:38.697699 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:38.697672    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qllbk\" (UniqueName: \"kubernetes.io/projected/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-kube-api-access-qllbk\") pod \"maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp\" (UID: \"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"
Sep 11 11:47:38.780966 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:38.780927    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:39.193052 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:39.192956    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-maas-api-tls\") pod \"maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp\" (UID: \"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"
Sep 11 11:47:39.195445 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:39.195420    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-maas-api-tls\") pod \"maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp\" (UID: \"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"
Sep 11 11:47:39.278047 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:39.278010    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:39.336121 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:39.336085    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98"]
Sep 11 11:47:39.339938 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:39.339917    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98"
Sep 11 11:47:39.346352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:39.346331    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-dbe0e7-dockercfg-fxbl6\""
Sep 11 11:47:39.346352 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:39.346334    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-dbe0e7\""
Sep 11 11:47:39.355392 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:39.355365    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98"]
Sep 11 11:47:39.384481 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:39.384448    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:39.394741 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:39.394713    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0993636b-9ec7-4e63-b5f0-856a883ad585-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98\" (UID: \"0993636b-9ec7-4e63-b5f0-856a883ad585\") " pod="openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98"
Sep 11 11:47:39.394892 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:39.394873    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72z99\" (UniqueName: \"kubernetes.io/projected/0993636b-9ec7-4e63-b5f0-856a883ad585-kube-api-access-72z99\") pod \"payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98\" (UID: \"0993636b-9ec7-4e63-b5f0-856a883ad585\") " pod="openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98"
Sep 11 11:47:39.404554 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:39.404530    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"
Sep 11 11:47:39.496099 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:39.496071    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72z99\" (UniqueName: \"kubernetes.io/projected/0993636b-9ec7-4e63-b5f0-856a883ad585-kube-api-access-72z99\") pod \"payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98\" (UID: \"0993636b-9ec7-4e63-b5f0-856a883ad585\") " pod="openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98"
Sep 11 11:47:39.496577 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:39.496119    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0993636b-9ec7-4e63-b5f0-856a883ad585-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98\" (UID: \"0993636b-9ec7-4e63-b5f0-856a883ad585\") " pod="openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98"
Sep 11 11:47:39.496769 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:39.496750    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0993636b-9ec7-4e63-b5f0-856a883ad585-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98\" (UID: \"0993636b-9ec7-4e63-b5f0-856a883ad585\") " pod="openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98"
Sep 11 11:47:39.504622 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:39.504577    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72z99\" (UniqueName: \"kubernetes.io/projected/0993636b-9ec7-4e63-b5f0-856a883ad585-kube-api-access-72z99\") pod \"payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98\" (UID: \"0993636b-9ec7-4e63-b5f0-856a883ad585\") " pod="openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98"
Sep 11 11:47:39.650552 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:39.650521    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98"
Sep 11 11:47:39.690712 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:39.690678    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"]
Sep 11 11:47:39.692225 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:47:39.692198    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c2fb2d3_5e2f_4a61_82e8_baa4b1ccb96a.slice/crio-a6c583df128a72b026b951235f6f0cf729b356a5f065b40f1a807a74e35c521e WatchSource:0}: Error finding container a6c583df128a72b026b951235f6f0cf729b356a5f065b40f1a807a74e35c521e: Status 404 returned error can't find the container with id a6c583df128a72b026b951235f6f0cf729b356a5f065b40f1a807a74e35c521e
Sep 11 11:47:39.961681 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:39.961659    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98"]
Sep 11 11:47:39.964101 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:47:39.964073    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0993636b_9ec7_4e63_b5f0_856a883ad585.slice/crio-2509ffb4db6b59cc775663868dad619335ff573c2b650e14c3a72ec32fde2933 WatchSource:0}: Error finding container 2509ffb4db6b59cc775663868dad619335ff573c2b650e14c3a72ec32fde2933: Status 404 returned error can't find the container with id 2509ffb4db6b59cc775663868dad619335ff573c2b650e14c3a72ec32fde2933
Sep 11 11:47:39.970364 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:39.970336    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp" event={"ID":"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a","Type":"ContainerStarted","Data":"a6c583df128a72b026b951235f6f0cf729b356a5f065b40f1a807a74e35c521e"}
Sep 11 11:47:39.972105 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:39.972075    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8023-4f16ea" event={"ID":"fb51ff15-a116-49c3-b7f6-2fea3ae4caea","Type":"ContainerStarted","Data":"a3de42fabd7748e1cbdc97831105ad0fa830342ae30d47c302de0a84aebfe2af"}
Sep 11 11:47:40.028827 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:40.028775    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8023-4f16ea" podStartSLOduration=2.626169911 podStartE2EDuration="5.028759328s" totalImagesPullingTime="2.402589417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:47:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:47:40.026960282 +0000 UTC m=+800.447566310" watchObservedRunningTime="2026-09-11 11:47:40.028759328 +0000 UTC m=+800.449365354"
Sep 11 11:47:40.728575 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:40.728495    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8023-4f16ea"]
Sep 11 11:47:40.734621 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:40.734595    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8023-4f16ea"]
Sep 11 11:47:40.870306 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:40.870273    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8023-218e8c"]
Sep 11 11:47:40.870690 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:40.870677    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb51ff15-a116-49c3-b7f6-2fea3ae4caea" containerName="test-curl-8023-4f16ea"
Sep 11 11:47:40.870734 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:40.870692    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb51ff15-a116-49c3-b7f6-2fea3ae4caea" containerName="test-curl-8023-4f16ea"
Sep 11 11:47:40.870782 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:40.870774    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb51ff15-a116-49c3-b7f6-2fea3ae4caea" containerName="test-curl-8023-4f16ea"
Sep 11 11:47:40.873834 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:40.873813    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8023-218e8c"
Sep 11 11:47:40.883768 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:40.883745    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8023-218e8c"]
Sep 11 11:47:40.908610 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:40.908580    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27ctl\" (UniqueName: \"kubernetes.io/projected/4da25e47-4fca-4945-83c0-4a90e27eb85c-kube-api-access-27ctl\") pod \"test-curl-8023-218e8c\" (UID: \"4da25e47-4fca-4945-83c0-4a90e27eb85c\") " pod="opendatahub/test-curl-8023-218e8c"
Sep 11 11:47:40.978537 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:40.978498    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp" event={"ID":"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a","Type":"ContainerStarted","Data":"a16224d73daa6007c0ad8770bb48764f6bfd513cb29238f5e26b0be1ef71874a"}
Sep 11 11:47:40.978753 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:40.978675    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"
Sep 11 11:47:40.980227 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:40.980203    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98" event={"ID":"0993636b-9ec7-4e63-b5f0-856a883ad585","Type":"ContainerStarted","Data":"872ecfcd011a27babe58b062e071dd1255515ae15d6eb9f7d863dae9d0914c6e"}
Sep 11 11:47:40.980344 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:40.980278    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98"
Sep 11 11:47:40.980344 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:40.980318    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98" event={"ID":"0993636b-9ec7-4e63-b5f0-856a883ad585","Type":"ContainerStarted","Data":"2509ffb4db6b59cc775663868dad619335ff573c2b650e14c3a72ec32fde2933"}
Sep 11 11:47:40.980425 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:47:40.980411    2575 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8023-4f16ea" container="test-curl-8023-4f16ea"
Sep 11 11:47:40.999358 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:40.998942    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp" podStartSLOduration=2.151860971 podStartE2EDuration="2.998922961s" totalImagesPullingTime="847.06199ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:47:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:47:40.998461289 +0000 UTC m=+801.419067317" watchObservedRunningTime="2026-09-11 11:47:40.998922961 +0000 UTC m=+801.419528990"
Sep 11 11:47:41.010065 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:41.010038    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27ctl\" (UniqueName: \"kubernetes.io/projected/4da25e47-4fca-4945-83c0-4a90e27eb85c-kube-api-access-27ctl\") pod \"test-curl-8023-218e8c\" (UID: \"4da25e47-4fca-4945-83c0-4a90e27eb85c\") " pod="opendatahub/test-curl-8023-218e8c"
Sep 11 11:47:41.022387 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:41.022362    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27ctl\" (UniqueName: \"kubernetes.io/projected/4da25e47-4fca-4945-83c0-4a90e27eb85c-kube-api-access-27ctl\") pod \"test-curl-8023-218e8c\" (UID: \"4da25e47-4fca-4945-83c0-4a90e27eb85c\") " pod="opendatahub/test-curl-8023-218e8c"
Sep 11 11:47:41.029420 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:41.029378    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98" podStartSLOduration=2.029365116 podStartE2EDuration="2.029365116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:47:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:47:41.026123971 +0000 UTC m=+801.446729997" watchObservedRunningTime="2026-09-11 11:47:41.029365116 +0000 UTC m=+801.449971142"
Sep 11 11:47:41.184788 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:41.184763    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8023-218e8c"
Sep 11 11:47:41.456423 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:41.456363    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8023-218e8c"]
Sep 11 11:47:41.458784 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:47:41.458754    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4da25e47_4fca_4945_83c0_4a90e27eb85c.slice/crio-7829de90db71ca8302247ec02359b95eb6a79d9d840dcf0ecb950b0cdd98183f WatchSource:0}: Error finding container 7829de90db71ca8302247ec02359b95eb6a79d9d840dcf0ecb950b0cdd98183f: Status 404 returned error can't find the container with id 7829de90db71ca8302247ec02359b95eb6a79d9d840dcf0ecb950b0cdd98183f
Sep 11 11:47:41.854954 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:41.854866    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:41.958704 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:41.958625    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:41.988275 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:41.988239    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8023-218e8c" event={"ID":"4da25e47-4fca-4945-83c0-4a90e27eb85c","Type":"ContainerStarted","Data":"e4fac36bbb17adfb94551f56942728721fbe7ce3c5c2b47868ac48689758def5"}
Sep 11 11:47:41.988468 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:41.988288    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8023-218e8c" event={"ID":"4da25e47-4fca-4945-83c0-4a90e27eb85c","Type":"ContainerStarted","Data":"7829de90db71ca8302247ec02359b95eb6a79d9d840dcf0ecb950b0cdd98183f"}
Sep 11 11:47:41.988608 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:41.988583    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8023-4f16ea" podUID="fb51ff15-a116-49c3-b7f6-2fea3ae4caea" containerName="test-curl-8023-4f16ea" containerID="cri-o://a3de42fabd7748e1cbdc97831105ad0fa830342ae30d47c302de0a84aebfe2af" gracePeriod=2
Sep 11 11:47:42.020581 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:42.020518    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8023-218e8c" podStartSLOduration=2.020501959 podStartE2EDuration="2.020501959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:47:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:47:42.018147349 +0000 UTC m=+802.438753377" watchObservedRunningTime="2026-09-11 11:47:42.020501959 +0000 UTC m=+802.441107987"
Sep 11 11:47:42.021834 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:47:42.021805    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8023-4f16ea\" is forbidden: User \"system:node:ip-10-0-135-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-16.ec2.internal' and this object" podUID="fb51ff15-a116-49c3-b7f6-2fea3ae4caea" pod="opendatahub/test-curl-8023-4f16ea"
Sep 11 11:47:42.114940 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:42.114839    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:42.492026 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:42.491986    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:42.748533 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:42.748498    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8023-218e8c"]
Sep 11 11:47:42.751210 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:42.751180    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8023-218e8c"]
Sep 11 11:47:42.993887 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:47:42.993857    2575 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8023-218e8c" container="test-curl-8023-218e8c"
Sep 11 11:47:43.073186 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:43.073096    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:43.278982 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:43.278928    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:43.653186 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:43.653148    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:43.843612 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:43.843578    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:43.999291 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:43.999244    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8023-218e8c" podUID="4da25e47-4fca-4945-83c0-4a90e27eb85c" containerName="test-curl-8023-218e8c" containerID="cri-o://e4fac36bbb17adfb94551f56942728721fbe7ce3c5c2b47868ac48689758def5" gracePeriod=2
Sep 11 11:47:44.001545 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:47:44.001511    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8023-218e8c\" is forbidden: User \"system:node:ip-10-0-135-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-16.ec2.internal' and this object" podUID="4da25e47-4fca-4945-83c0-4a90e27eb85c" pod="opendatahub/test-curl-8023-218e8c"
Sep 11 11:47:44.162715 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:44.162695    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8023-4f16ea"
Sep 11 11:47:44.240408 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:44.240383    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8zgw\" (UniqueName: \"kubernetes.io/projected/fb51ff15-a116-49c3-b7f6-2fea3ae4caea-kube-api-access-d8zgw\") pod \"fb51ff15-a116-49c3-b7f6-2fea3ae4caea\" (UID: \"fb51ff15-a116-49c3-b7f6-2fea3ae4caea\") "
Sep 11 11:47:44.242430 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:44.242368    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb51ff15-a116-49c3-b7f6-2fea3ae4caea-kube-api-access-d8zgw" (OuterVolumeSpecName: "kube-api-access-d8zgw") pod "fb51ff15-a116-49c3-b7f6-2fea3ae4caea" (UID: "fb51ff15-a116-49c3-b7f6-2fea3ae4caea"). InnerVolumeSpecName "kube-api-access-d8zgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:47:44.341542 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:44.341511    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8zgw\" (UniqueName: \"kubernetes.io/projected/fb51ff15-a116-49c3-b7f6-2fea3ae4caea-kube-api-access-d8zgw\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:47:44.793281 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:44.793237    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:44.881527 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:44.881494    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:45.006686 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:45.006621    2575 generic.go:356] "Generic (PLEG): container finished" podID="fb51ff15-a116-49c3-b7f6-2fea3ae4caea" containerID="a3de42fabd7748e1cbdc97831105ad0fa830342ae30d47c302de0a84aebfe2af" exitCode=137
Sep 11 11:47:45.006686 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:45.006685    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8023-4f16ea"
Sep 11 11:47:45.007144 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:45.006727    2575 scope.go:117] "RemoveContainer" containerID="a3de42fabd7748e1cbdc97831105ad0fa830342ae30d47c302de0a84aebfe2af"
Sep 11 11:47:45.015754 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:45.015729    2575 scope.go:117] "RemoveContainer" containerID="a3de42fabd7748e1cbdc97831105ad0fa830342ae30d47c302de0a84aebfe2af"
Sep 11 11:47:45.016056 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:47:45.016039    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3de42fabd7748e1cbdc97831105ad0fa830342ae30d47c302de0a84aebfe2af\": container with ID starting with a3de42fabd7748e1cbdc97831105ad0fa830342ae30d47c302de0a84aebfe2af not found: ID does not exist" containerID="a3de42fabd7748e1cbdc97831105ad0fa830342ae30d47c302de0a84aebfe2af"
Sep 11 11:47:45.016127 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:45.016065    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3de42fabd7748e1cbdc97831105ad0fa830342ae30d47c302de0a84aebfe2af"} err="failed to get container status \"a3de42fabd7748e1cbdc97831105ad0fa830342ae30d47c302de0a84aebfe2af\": rpc error: code = NotFound desc = could not find container \"a3de42fabd7748e1cbdc97831105ad0fa830342ae30d47c302de0a84aebfe2af\": container with ID starting with a3de42fabd7748e1cbdc97831105ad0fa830342ae30d47c302de0a84aebfe2af not found: ID does not exist"
Sep 11 11:47:45.018911 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:47:45.018888    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8023-4f16ea\" is forbidden: User \"system:node:ip-10-0-135-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-16.ec2.internal' and this object" podUID="fb51ff15-a116-49c3-b7f6-2fea3ae4caea" pod="opendatahub/test-curl-8023-4f16ea"
Sep 11 11:47:45.913999 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:45.913963    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"]
Sep 11 11:47:45.914428 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:45.914416    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4da25e47-4fca-4945-83c0-4a90e27eb85c" containerName="test-curl-8023-218e8c"
Sep 11 11:47:45.914492 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:45.914431    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4da25e47-4fca-4945-83c0-4a90e27eb85c" containerName="test-curl-8023-218e8c"
Sep 11 11:47:45.914534 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:45.914512    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4da25e47-4fca-4945-83c0-4a90e27eb85c" containerName="test-curl-8023-218e8c"
Sep 11 11:47:45.918896 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:45.918873    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"
Sep 11 11:47:45.922090 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:45.922066    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-bddd3ec1-serving-cert\""
Sep 11 11:47:45.947370 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:45.947346    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"]
Sep 11 11:47:45.956957 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:45.956934    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2be5fb11-70a2-4d39-bb01-b00fcab1903c-maas-api-tls\") pod \"maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9\" (UID: \"2be5fb11-70a2-4d39-bb01-b00fcab1903c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"
Sep 11 11:47:45.957101 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:45.957040    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2be5fb11-70a2-4d39-bb01-b00fcab1903c-openshift-service-ca\") pod \"maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9\" (UID: \"2be5fb11-70a2-4d39-bb01-b00fcab1903c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"
Sep 11 11:47:45.957101 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:45.957074    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j257z\" (UniqueName: \"kubernetes.io/projected/2be5fb11-70a2-4d39-bb01-b00fcab1903c-kube-api-access-j257z\") pod \"maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9\" (UID: \"2be5fb11-70a2-4d39-bb01-b00fcab1903c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"
Sep 11 11:47:45.957211 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:45.957138    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2be5fb11-70a2-4d39-bb01-b00fcab1903c-metrics-certs\") pod \"maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9\" (UID: \"2be5fb11-70a2-4d39-bb01-b00fcab1903c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"
Sep 11 11:47:45.962017 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:47:45.961988    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8023-4f16ea\" is forbidden: User \"system:node:ip-10-0-135-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-16.ec2.internal' and this object" podUID="fb51ff15-a116-49c3-b7f6-2fea3ae4caea" pod="opendatahub/test-curl-8023-4f16ea"
Sep 11 11:47:45.999714 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:45.999680    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:46.058795 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:46.058766    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2be5fb11-70a2-4d39-bb01-b00fcab1903c-maas-api-tls\") pod \"maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9\" (UID: \"2be5fb11-70a2-4d39-bb01-b00fcab1903c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"
Sep 11 11:47:46.059261 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:46.058845    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2be5fb11-70a2-4d39-bb01-b00fcab1903c-openshift-service-ca\") pod \"maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9\" (UID: \"2be5fb11-70a2-4d39-bb01-b00fcab1903c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"
Sep 11 11:47:46.059261 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:46.058877    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j257z\" (UniqueName: \"kubernetes.io/projected/2be5fb11-70a2-4d39-bb01-b00fcab1903c-kube-api-access-j257z\") pod \"maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9\" (UID: \"2be5fb11-70a2-4d39-bb01-b00fcab1903c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"
Sep 11 11:47:46.059261 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:46.058928    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2be5fb11-70a2-4d39-bb01-b00fcab1903c-metrics-certs\") pod \"maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9\" (UID: \"2be5fb11-70a2-4d39-bb01-b00fcab1903c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"
Sep 11 11:47:46.059261 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:47:46.058949    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-serving-cert: secret "maas-api-e2e-ait-bddd3ec1-serving-cert" not found
Sep 11 11:47:46.059261 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:47:46.059026    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2be5fb11-70a2-4d39-bb01-b00fcab1903c-maas-api-tls podName:2be5fb11-70a2-4d39-bb01-b00fcab1903c nodeName:}" failed. No retries permitted until 2026-09-11 11:47:46.559003442 +0000 UTC m=+806.979609471 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2be5fb11-70a2-4d39-bb01-b00fcab1903c-maas-api-tls") pod "maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9" (UID: "2be5fb11-70a2-4d39-bb01-b00fcab1903c") : secret "maas-api-e2e-ait-bddd3ec1-serving-cert" not found
Sep 11 11:47:46.059684 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:46.059664    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2be5fb11-70a2-4d39-bb01-b00fcab1903c-openshift-service-ca\") pod \"maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9\" (UID: \"2be5fb11-70a2-4d39-bb01-b00fcab1903c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"
Sep 11 11:47:46.061450 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:46.061433    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2be5fb11-70a2-4d39-bb01-b00fcab1903c-metrics-certs\") pod \"maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9\" (UID: \"2be5fb11-70a2-4d39-bb01-b00fcab1903c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"
Sep 11 11:47:46.075812 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:46.075788    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j257z\" (UniqueName: \"kubernetes.io/projected/2be5fb11-70a2-4d39-bb01-b00fcab1903c-kube-api-access-j257z\") pod \"maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9\" (UID: \"2be5fb11-70a2-4d39-bb01-b00fcab1903c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"
Sep 11 11:47:46.084200 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:46.084171    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb51ff15-a116-49c3-b7f6-2fea3ae4caea" path="/var/lib/kubelet/pods/fb51ff15-a116-49c3-b7f6-2fea3ae4caea/volumes"
Sep 11 11:47:46.109469 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:46.109433    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:46.146857 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:46.146833    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8023-218e8c"
Sep 11 11:47:46.260868 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:46.260839    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27ctl\" (UniqueName: \"kubernetes.io/projected/4da25e47-4fca-4945-83c0-4a90e27eb85c-kube-api-access-27ctl\") pod \"4da25e47-4fca-4945-83c0-4a90e27eb85c\" (UID: \"4da25e47-4fca-4945-83c0-4a90e27eb85c\") "
Sep 11 11:47:46.263001 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:46.262970    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4da25e47-4fca-4945-83c0-4a90e27eb85c-kube-api-access-27ctl" (OuterVolumeSpecName: "kube-api-access-27ctl") pod "4da25e47-4fca-4945-83c0-4a90e27eb85c" (UID: "4da25e47-4fca-4945-83c0-4a90e27eb85c"). InnerVolumeSpecName "kube-api-access-27ctl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:47:46.362183 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:46.362152    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27ctl\" (UniqueName: \"kubernetes.io/projected/4da25e47-4fca-4945-83c0-4a90e27eb85c-kube-api-access-27ctl\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:47:46.430858 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:46.430829    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:46.555951 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:46.555868    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:46.564319 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:46.564289    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2be5fb11-70a2-4d39-bb01-b00fcab1903c-maas-api-tls\") pod \"maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9\" (UID: \"2be5fb11-70a2-4d39-bb01-b00fcab1903c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"
Sep 11 11:47:46.567400 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:46.567371    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2be5fb11-70a2-4d39-bb01-b00fcab1903c-maas-api-tls\") pod \"maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9\" (UID: \"2be5fb11-70a2-4d39-bb01-b00fcab1903c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"
Sep 11 11:47:46.675872 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:46.675824    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:46.831275 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:46.831196    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"
Sep 11 11:47:46.994447 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:46.994423    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"
Sep 11 11:47:47.020831 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:47.020803    2575 generic.go:356] "Generic (PLEG): container finished" podID="4da25e47-4fca-4945-83c0-4a90e27eb85c" containerID="e4fac36bbb17adfb94551f56942728721fbe7ce3c5c2b47868ac48689758def5" exitCode=137
Sep 11 11:47:47.020972 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:47.020839    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8023-218e8c"
Sep 11 11:47:47.020972 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:47.020887    2575 scope.go:117] "RemoveContainer" containerID="e4fac36bbb17adfb94551f56942728721fbe7ce3c5c2b47868ac48689758def5"
Sep 11 11:47:47.030003 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:47.029989    2575 scope.go:117] "RemoveContainer" containerID="e4fac36bbb17adfb94551f56942728721fbe7ce3c5c2b47868ac48689758def5"
Sep 11 11:47:47.030264 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:47:47.030246    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4fac36bbb17adfb94551f56942728721fbe7ce3c5c2b47868ac48689758def5\": container with ID starting with e4fac36bbb17adfb94551f56942728721fbe7ce3c5c2b47868ac48689758def5 not found: ID does not exist" containerID="e4fac36bbb17adfb94551f56942728721fbe7ce3c5c2b47868ac48689758def5"
Sep 11 11:47:47.030318 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:47.030271    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4fac36bbb17adfb94551f56942728721fbe7ce3c5c2b47868ac48689758def5"} err="failed to get container status \"e4fac36bbb17adfb94551f56942728721fbe7ce3c5c2b47868ac48689758def5\": rpc error: code = NotFound desc = could not find container \"e4fac36bbb17adfb94551f56942728721fbe7ce3c5c2b47868ac48689758def5\": container with ID starting with e4fac36bbb17adfb94551f56942728721fbe7ce3c5c2b47868ac48689758def5 not found: ID does not exist"
Sep 11 11:47:47.030750 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:47:47.030729    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8023-218e8c\" is forbidden: User \"system:node:ip-10-0-135-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-16.ec2.internal' and this object" podUID="4da25e47-4fca-4945-83c0-4a90e27eb85c" pod="opendatahub/test-curl-8023-218e8c"
Sep 11 11:47:47.108617 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:47.108584    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"]
Sep 11 11:47:47.109335 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:47:47.109294    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2be5fb11_70a2_4d39_bb01_b00fcab1903c.slice/crio-b99e226ad4003aad7f6afff3420d66523de143e682b67251aca83b534e45b52d WatchSource:0}: Error finding container b99e226ad4003aad7f6afff3420d66523de143e682b67251aca83b534e45b52d: Status 404 returned error can't find the container with id b99e226ad4003aad7f6afff3420d66523de143e682b67251aca83b534e45b52d
Sep 11 11:47:47.317388 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:47.317353    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:47.490642 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:47.490599    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:48.029691 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:48.029649    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9" event={"ID":"2be5fb11-70a2-4d39-bb01-b00fcab1903c","Type":"ContainerStarted","Data":"97c4b2afbb6bb124d9ce8d8e6b4cfd054c970c2ad529a2495e9c5778a8226d63"}
Sep 11 11:47:48.029877 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:48.029718    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9" event={"ID":"2be5fb11-70a2-4d39-bb01-b00fcab1903c","Type":"ContainerStarted","Data":"b99e226ad4003aad7f6afff3420d66523de143e682b67251aca83b534e45b52d"}
Sep 11 11:47:48.030515 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:48.030486    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"
Sep 11 11:47:48.056508 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:48.056459    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9" podStartSLOduration=2.582176441 podStartE2EDuration="3.05643764s" totalImagesPullingTime="474.261199ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:47:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:47:48.05445567 +0000 UTC m=+808.475061698" watchObservedRunningTime="2026-09-11 11:47:48.05643764 +0000 UTC m=+808.477043668"
Sep 11 11:47:48.058303 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:47:48.058274    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8023-218e8c\" is forbidden: User \"system:node:ip-10-0-135-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-16.ec2.internal' and this object" podUID="4da25e47-4fca-4945-83c0-4a90e27eb85c" pod="opendatahub/test-curl-8023-218e8c"
Sep 11 11:47:48.083327 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:48.083294    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4da25e47-4fca-4945-83c0-4a90e27eb85c" path="/var/lib/kubelet/pods/4da25e47-4fca-4945-83c0-4a90e27eb85c/volumes"
Sep 11 11:47:48.837199 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:48.837163    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"]
Sep 11 11:47:48.841528 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:48.841503    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"
Sep 11 11:47:48.843714 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:48.843688    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-5a5ab7-serving-cert\""
Sep 11 11:47:48.860857 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:48.857128    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"]
Sep 11 11:47:48.881130 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:48.881095    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:48.886311 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:48.886286    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa73d8ab-d829-47d9-84d2-f6fd20c954af-maas-api-tls\") pod \"maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9\" (UID: \"fa73d8ab-d829-47d9-84d2-f6fd20c954af\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"
Sep 11 11:47:48.886451 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:48.886319    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa73d8ab-d829-47d9-84d2-f6fd20c954af-metrics-certs\") pod \"maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9\" (UID: \"fa73d8ab-d829-47d9-84d2-f6fd20c954af\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"
Sep 11 11:47:48.886451 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:48.886434    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa73d8ab-d829-47d9-84d2-f6fd20c954af-openshift-service-ca\") pod \"maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9\" (UID: \"fa73d8ab-d829-47d9-84d2-f6fd20c954af\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"
Sep 11 11:47:48.886551 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:48.886466    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmzdd\" (UniqueName: \"kubernetes.io/projected/fa73d8ab-d829-47d9-84d2-f6fd20c954af-kube-api-access-lmzdd\") pod \"maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9\" (UID: \"fa73d8ab-d829-47d9-84d2-f6fd20c954af\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"
Sep 11 11:47:48.985140 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:48.985100    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:48.987298 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:48.987275    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa73d8ab-d829-47d9-84d2-f6fd20c954af-openshift-service-ca\") pod \"maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9\" (UID: \"fa73d8ab-d829-47d9-84d2-f6fd20c954af\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"
Sep 11 11:47:48.987433 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:48.987303    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmzdd\" (UniqueName: \"kubernetes.io/projected/fa73d8ab-d829-47d9-84d2-f6fd20c954af-kube-api-access-lmzdd\") pod \"maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9\" (UID: \"fa73d8ab-d829-47d9-84d2-f6fd20c954af\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"
Sep 11 11:47:48.987433 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:48.987394    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa73d8ab-d829-47d9-84d2-f6fd20c954af-maas-api-tls\") pod \"maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9\" (UID: \"fa73d8ab-d829-47d9-84d2-f6fd20c954af\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"
Sep 11 11:47:48.987522 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:47:48.987488    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-serving-cert: secret "maas-api-e2e-autores-5a5ab7-serving-cert" not found
Sep 11 11:47:48.987580 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:48.987530    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa73d8ab-d829-47d9-84d2-f6fd20c954af-metrics-certs\") pod \"maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9\" (UID: \"fa73d8ab-d829-47d9-84d2-f6fd20c954af\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"
Sep 11 11:47:48.987580 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:47:48.987550    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa73d8ab-d829-47d9-84d2-f6fd20c954af-maas-api-tls podName:fa73d8ab-d829-47d9-84d2-f6fd20c954af nodeName:}" failed. No retries permitted until 2026-09-11 11:47:49.487529713 +0000 UTC m=+809.908135718 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fa73d8ab-d829-47d9-84d2-f6fd20c954af-maas-api-tls") pod "maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9" (UID: "fa73d8ab-d829-47d9-84d2-f6fd20c954af") : secret "maas-api-e2e-autores-5a5ab7-serving-cert" not found
Sep 11 11:47:48.988006 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:48.987989    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa73d8ab-d829-47d9-84d2-f6fd20c954af-openshift-service-ca\") pod \"maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9\" (UID: \"fa73d8ab-d829-47d9-84d2-f6fd20c954af\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"
Sep 11 11:47:48.990050 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:48.990029    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa73d8ab-d829-47d9-84d2-f6fd20c954af-metrics-certs\") pod \"maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9\" (UID: \"fa73d8ab-d829-47d9-84d2-f6fd20c954af\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"
Sep 11 11:47:48.999798 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:48.999774    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmzdd\" (UniqueName: \"kubernetes.io/projected/fa73d8ab-d829-47d9-84d2-f6fd20c954af-kube-api-access-lmzdd\") pod \"maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9\" (UID: \"fa73d8ab-d829-47d9-84d2-f6fd20c954af\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"
Sep 11 11:47:49.185417 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:49.185326    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:49.389116 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:49.389076    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:49.490731 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:49.490706    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:49.491871 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:49.491848    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa73d8ab-d829-47d9-84d2-f6fd20c954af-maas-api-tls\") pod \"maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9\" (UID: \"fa73d8ab-d829-47d9-84d2-f6fd20c954af\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"
Sep 11 11:47:49.494288 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:49.494261    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa73d8ab-d829-47d9-84d2-f6fd20c954af-maas-api-tls\") pod \"maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9\" (UID: \"fa73d8ab-d829-47d9-84d2-f6fd20c954af\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"
Sep 11 11:47:49.757786 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:49.757703    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"
Sep 11 11:47:49.879853 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:49.879804    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:50.032482 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:50.032461    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"]
Sep 11 11:47:50.034118 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:47:50.034091    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa73d8ab_d829_47d9_84d2_f6fd20c954af.slice/crio-64e8cd6bf33794b7cf2c4f846750832ab7098080ff7b7a3dab1608d519887d24 WatchSource:0}: Error finding container 64e8cd6bf33794b7cf2c4f846750832ab7098080ff7b7a3dab1608d519887d24: Status 404 returned error can't find the container with id 64e8cd6bf33794b7cf2c4f846750832ab7098080ff7b7a3dab1608d519887d24
Sep 11 11:47:50.045980 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:50.045955    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9" event={"ID":"fa73d8ab-d829-47d9-84d2-f6fd20c954af","Type":"ContainerStarted","Data":"64e8cd6bf33794b7cf2c4f846750832ab7098080ff7b7a3dab1608d519887d24"}
Sep 11 11:47:50.703350 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:50.703316    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:51.053952 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:51.053920    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9" event={"ID":"fa73d8ab-d829-47d9-84d2-f6fd20c954af","Type":"ContainerStarted","Data":"9e4904525afc89e02789e9e6c9d1f3ebf6af325e4edf9ddbc394a865985922ba"}
Sep 11 11:47:51.054395 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:51.053993    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"
Sep 11 11:47:51.075679 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:51.075616    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9" podStartSLOduration=2.628104587 podStartE2EDuration="3.075600421s" totalImagesPullingTime="447.495834ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:47:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:47:51.074057119 +0000 UTC m=+811.494663137" watchObservedRunningTime="2026-09-11 11:47:51.075600421 +0000 UTC m=+811.496206448"
Sep 11 11:47:51.990819 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:51.990790    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98"
Sep 11 11:47:52.702292 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:52.702261    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:52.934298 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:52.934268    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:53.202277 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:53.202242    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:53.364620 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:53.364586    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:55.051174 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:55.051148    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"
Sep 11 11:47:55.748280 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:55.748238    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:57.064358 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:57.064331    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"
Sep 11 11:47:57.471767 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:57.471668    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:58.972556 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:58.972506    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:59.074458 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:59.074428    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:59.180590 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:59.180552    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:59.307167 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:59.307140    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:47:59.765688 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:47:59.765656    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:02.158640 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:02.158604    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:02.479133 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:02.479046    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:02.773723 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:02.773690    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:02.873971 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:02.873939    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:05.515265 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:05.515235    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:05.580159 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:05.580126    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:06.360263 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:06.360209    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:06.601210 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:06.601169    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:07.463159 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:07.463129    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"]
Sep 11 11:48:07.470526 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:07.470504    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"
Sep 11 11:48:07.473546 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:07.473516    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-c6bcf5-serving-cert\""
Sep 11 11:48:07.478804 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:07.478241    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"]
Sep 11 11:48:07.567069 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:07.567032    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/464427ef-23dc-471f-a0e1-7fa6c84625bf-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52\" (UID: \"464427ef-23dc-471f-a0e1-7fa6c84625bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"
Sep 11 11:48:07.567240 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:07.567083    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/464427ef-23dc-471f-a0e1-7fa6c84625bf-metrics-certs\") pod \"maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52\" (UID: \"464427ef-23dc-471f-a0e1-7fa6c84625bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"
Sep 11 11:48:07.567240 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:07.567150    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6264\" (UniqueName: \"kubernetes.io/projected/464427ef-23dc-471f-a0e1-7fa6c84625bf-kube-api-access-g6264\") pod \"maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52\" (UID: \"464427ef-23dc-471f-a0e1-7fa6c84625bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"
Sep 11 11:48:07.567240 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:07.567179    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/464427ef-23dc-471f-a0e1-7fa6c84625bf-maas-api-tls\") pod \"maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52\" (UID: \"464427ef-23dc-471f-a0e1-7fa6c84625bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"
Sep 11 11:48:07.665987 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:07.665952    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:07.668133 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:07.668111    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/464427ef-23dc-471f-a0e1-7fa6c84625bf-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52\" (UID: \"464427ef-23dc-471f-a0e1-7fa6c84625bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"
Sep 11 11:48:07.668229 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:07.668148    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/464427ef-23dc-471f-a0e1-7fa6c84625bf-metrics-certs\") pod \"maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52\" (UID: \"464427ef-23dc-471f-a0e1-7fa6c84625bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"
Sep 11 11:48:07.668229 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:07.668184    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6264\" (UniqueName: \"kubernetes.io/projected/464427ef-23dc-471f-a0e1-7fa6c84625bf-kube-api-access-g6264\") pod \"maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52\" (UID: \"464427ef-23dc-471f-a0e1-7fa6c84625bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"
Sep 11 11:48:07.668229 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:07.668207    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/464427ef-23dc-471f-a0e1-7fa6c84625bf-maas-api-tls\") pod \"maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52\" (UID: \"464427ef-23dc-471f-a0e1-7fa6c84625bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"
Sep 11 11:48:07.668374 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:48:07.668340    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-serving-cert: secret "maas-api-e2e-inf-b-c6bcf5-serving-cert" not found
Sep 11 11:48:07.668428 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:48:07.668410    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/464427ef-23dc-471f-a0e1-7fa6c84625bf-maas-api-tls podName:464427ef-23dc-471f-a0e1-7fa6c84625bf nodeName:}" failed. No retries permitted until 2026-09-11 11:48:08.168388668 +0000 UTC m=+828.588994675 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/464427ef-23dc-471f-a0e1-7fa6c84625bf-maas-api-tls") pod "maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52" (UID: "464427ef-23dc-471f-a0e1-7fa6c84625bf") : secret "maas-api-e2e-inf-b-c6bcf5-serving-cert" not found
Sep 11 11:48:07.668926 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:07.668899    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/464427ef-23dc-471f-a0e1-7fa6c84625bf-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52\" (UID: \"464427ef-23dc-471f-a0e1-7fa6c84625bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"
Sep 11 11:48:07.671266 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:07.671246    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/464427ef-23dc-471f-a0e1-7fa6c84625bf-metrics-certs\") pod \"maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52\" (UID: \"464427ef-23dc-471f-a0e1-7fa6c84625bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"
Sep 11 11:48:07.680173 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:07.680146    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6264\" (UniqueName: \"kubernetes.io/projected/464427ef-23dc-471f-a0e1-7fa6c84625bf-kube-api-access-g6264\") pod \"maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52\" (UID: \"464427ef-23dc-471f-a0e1-7fa6c84625bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"
Sep 11 11:48:07.765506 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:07.765474    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:08.048459 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:08.048369    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:08.147210 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:08.147173    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:08.173695 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:08.173668    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/464427ef-23dc-471f-a0e1-7fa6c84625bf-maas-api-tls\") pod \"maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52\" (UID: \"464427ef-23dc-471f-a0e1-7fa6c84625bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"
Sep 11 11:48:08.176210 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:08.176184    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/464427ef-23dc-471f-a0e1-7fa6c84625bf-maas-api-tls\") pod \"maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52\" (UID: \"464427ef-23dc-471f-a0e1-7fa6c84625bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"
Sep 11 11:48:08.205089 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:08.205066    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7"]
Sep 11 11:48:08.209065 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:08.209046    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7"
Sep 11 11:48:08.211541 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:08.211517    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-c6bcf5-dockercfg-d6q22\""
Sep 11 11:48:08.211849 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:08.211834    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-c6bcf5\""
Sep 11 11:48:08.226895 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:08.226868    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7"]
Sep 11 11:48:08.273877 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:08.273835    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:08.375616 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:08.375536    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c233f4b-1aa2-404f-86d4-15fad1d74569-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7\" (UID: \"7c233f4b-1aa2-404f-86d4-15fad1d74569\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7"
Sep 11 11:48:08.375616 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:08.375581    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5dp4\" (UniqueName: \"kubernetes.io/projected/7c233f4b-1aa2-404f-86d4-15fad1d74569-kube-api-access-z5dp4\") pod \"payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7\" (UID: \"7c233f4b-1aa2-404f-86d4-15fad1d74569\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7"
Sep 11 11:48:08.386309 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:08.386271    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"
Sep 11 11:48:08.476620 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:08.476582    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5dp4\" (UniqueName: \"kubernetes.io/projected/7c233f4b-1aa2-404f-86d4-15fad1d74569-kube-api-access-z5dp4\") pod \"payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7\" (UID: \"7c233f4b-1aa2-404f-86d4-15fad1d74569\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7"
Sep 11 11:48:08.476816 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:08.476799    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c233f4b-1aa2-404f-86d4-15fad1d74569-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7\" (UID: \"7c233f4b-1aa2-404f-86d4-15fad1d74569\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7"
Sep 11 11:48:08.477735 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:08.477713    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c233f4b-1aa2-404f-86d4-15fad1d74569-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7\" (UID: \"7c233f4b-1aa2-404f-86d4-15fad1d74569\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7"
Sep 11 11:48:08.505069 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:08.505041    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5dp4\" (UniqueName: \"kubernetes.io/projected/7c233f4b-1aa2-404f-86d4-15fad1d74569-kube-api-access-z5dp4\") pod \"payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7\" (UID: \"7c233f4b-1aa2-404f-86d4-15fad1d74569\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7"
Sep 11 11:48:08.519910 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:08.519887    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7"
Sep 11 11:48:08.668054 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:08.668027    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"]
Sep 11 11:48:08.669625 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:48:08.669601    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod464427ef_23dc_471f_a0e1_7fa6c84625bf.slice/crio-4c2c985d6ae0b0199533cb2366350599ce9b0efa1d63abf5ab28fd60ca19ba79 WatchSource:0}: Error finding container 4c2c985d6ae0b0199533cb2366350599ce9b0efa1d63abf5ab28fd60ca19ba79: Status 404 returned error can't find the container with id 4c2c985d6ae0b0199533cb2366350599ce9b0efa1d63abf5ab28fd60ca19ba79
Sep 11 11:48:08.805936 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:08.805909    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7"]
Sep 11 11:48:08.808683 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:48:08.808627    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c233f4b_1aa2_404f_86d4_15fad1d74569.slice/crio-b53bb1c26eb3533ee5f3b54fd515cc07cb3e3c755cb47efc447f3bf92b4fa088 WatchSource:0}: Error finding container b53bb1c26eb3533ee5f3b54fd515cc07cb3e3c755cb47efc447f3bf92b4fa088: Status 404 returned error can't find the container with id b53bb1c26eb3533ee5f3b54fd515cc07cb3e3c755cb47efc447f3bf92b4fa088
Sep 11 11:48:09.155782 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:09.155746    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52" event={"ID":"464427ef-23dc-471f-a0e1-7fa6c84625bf","Type":"ContainerStarted","Data":"e0ee0d639eab786215d73221f204d7668d2c37651f6761ce08e4124dbdc41ebe"}
Sep 11 11:48:09.155972 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:09.155812    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52" event={"ID":"464427ef-23dc-471f-a0e1-7fa6c84625bf","Type":"ContainerStarted","Data":"4c2c985d6ae0b0199533cb2366350599ce9b0efa1d63abf5ab28fd60ca19ba79"}
Sep 11 11:48:09.155972 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:09.155848    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"
Sep 11 11:48:09.157418 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:09.157397    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7" event={"ID":"7c233f4b-1aa2-404f-86d4-15fad1d74569","Type":"ContainerStarted","Data":"42bfc11f890199e255ef3bf0481d935afb974845c93e6204ab5771ebe2df1301"}
Sep 11 11:48:09.157534 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:09.157435    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7" event={"ID":"7c233f4b-1aa2-404f-86d4-15fad1d74569","Type":"ContainerStarted","Data":"b53bb1c26eb3533ee5f3b54fd515cc07cb3e3c755cb47efc447f3bf92b4fa088"}
Sep 11 11:48:09.157534 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:09.157455    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7"
Sep 11 11:48:09.181699 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:09.181616    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52" podStartSLOduration=1.838814602 podStartE2EDuration="2.181595457s" totalImagesPullingTime="342.780855ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:48:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:09.181003182 +0000 UTC m=+829.601609208" watchObservedRunningTime="2026-09-11 11:48:09.181595457 +0000 UTC m=+829.602201488"
Sep 11 11:48:09.301599 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:09.301482    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:10.100168 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:10.100121    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:11.194513 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:11.194455    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:11.993597 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:11.993569    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:13.004644 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:13.004600    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:15.168462 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:15.168436    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"
Sep 11 11:48:15.200641 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:15.200593    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7" podStartSLOduration=7.20057864 podStartE2EDuration="7.20057864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:48:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:09.228022913 +0000 UTC m=+829.648628951" watchObservedRunningTime="2026-09-11 11:48:15.20057864 +0000 UTC m=+835.621184667"
Sep 11 11:48:15.476800 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:15.476693    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:17.371948 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:17.371915    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:20.165393 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:20.165366    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7"
Sep 11 11:48:22.451389 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:22.451361    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:26.599723 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:26.599694    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"]
Sep 11 11:48:26.600172 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:26.599892    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9" podUID="2be5fb11-70a2-4d39-bb01-b00fcab1903c" containerName="maas-api" containerID="cri-o://97c4b2afbb6bb124d9ce8d8e6b4cfd054c970c2ad529a2495e9c5778a8226d63" gracePeriod=30
Sep 11 11:48:26.670864 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:26.670835    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:26.862085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:26.862002    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:26.862266 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:26.862255    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"
Sep 11 11:48:26.948755 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:26.948723    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j257z\" (UniqueName: \"kubernetes.io/projected/2be5fb11-70a2-4d39-bb01-b00fcab1903c-kube-api-access-j257z\") pod \"2be5fb11-70a2-4d39-bb01-b00fcab1903c\" (UID: \"2be5fb11-70a2-4d39-bb01-b00fcab1903c\") "
Sep 11 11:48:26.948908 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:26.948782    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2be5fb11-70a2-4d39-bb01-b00fcab1903c-metrics-certs\") pod \"2be5fb11-70a2-4d39-bb01-b00fcab1903c\" (UID: \"2be5fb11-70a2-4d39-bb01-b00fcab1903c\") "
Sep 11 11:48:26.948908 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:26.948811    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2be5fb11-70a2-4d39-bb01-b00fcab1903c-openshift-service-ca\") pod \"2be5fb11-70a2-4d39-bb01-b00fcab1903c\" (UID: \"2be5fb11-70a2-4d39-bb01-b00fcab1903c\") "
Sep 11 11:48:26.948908 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:26.948861    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2be5fb11-70a2-4d39-bb01-b00fcab1903c-maas-api-tls\") pod \"2be5fb11-70a2-4d39-bb01-b00fcab1903c\" (UID: \"2be5fb11-70a2-4d39-bb01-b00fcab1903c\") "
Sep 11 11:48:26.949174 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:26.949132    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2be5fb11-70a2-4d39-bb01-b00fcab1903c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2be5fb11-70a2-4d39-bb01-b00fcab1903c" (UID: "2be5fb11-70a2-4d39-bb01-b00fcab1903c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:48:26.951322 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:26.951290    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2be5fb11-70a2-4d39-bb01-b00fcab1903c-kube-api-access-j257z" (OuterVolumeSpecName: "kube-api-access-j257z") pod "2be5fb11-70a2-4d39-bb01-b00fcab1903c" (UID: "2be5fb11-70a2-4d39-bb01-b00fcab1903c"). InnerVolumeSpecName "kube-api-access-j257z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:48:26.951446 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:26.951318    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2be5fb11-70a2-4d39-bb01-b00fcab1903c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2be5fb11-70a2-4d39-bb01-b00fcab1903c" (UID: "2be5fb11-70a2-4d39-bb01-b00fcab1903c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:48:26.951446 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:26.951333    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2be5fb11-70a2-4d39-bb01-b00fcab1903c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2be5fb11-70a2-4d39-bb01-b00fcab1903c" (UID: "2be5fb11-70a2-4d39-bb01-b00fcab1903c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:48:27.050466 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:27.050429    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2be5fb11-70a2-4d39-bb01-b00fcab1903c-metrics-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:48:27.050466 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:27.050457    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2be5fb11-70a2-4d39-bb01-b00fcab1903c-openshift-service-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:48:27.050466 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:27.050470    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2be5fb11-70a2-4d39-bb01-b00fcab1903c-maas-api-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:48:27.050747 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:27.050486    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j257z\" (UniqueName: \"kubernetes.io/projected/2be5fb11-70a2-4d39-bb01-b00fcab1903c-kube-api-access-j257z\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:48:27.053286 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:27.053258    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:27.168126 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:27.168027    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:27.243173 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:27.243136    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:27.251967 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:27.251940    2575 generic.go:356] "Generic (PLEG): container finished" podID="2be5fb11-70a2-4d39-bb01-b00fcab1903c" containerID="97c4b2afbb6bb124d9ce8d8e6b4cfd054c970c2ad529a2495e9c5778a8226d63" exitCode=0
Sep 11 11:48:27.252136 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:27.251984    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"
Sep 11 11:48:27.252136 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:27.252013    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9" event={"ID":"2be5fb11-70a2-4d39-bb01-b00fcab1903c","Type":"ContainerDied","Data":"97c4b2afbb6bb124d9ce8d8e6b4cfd054c970c2ad529a2495e9c5778a8226d63"}
Sep 11 11:48:27.252136 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:27.252074    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9" event={"ID":"2be5fb11-70a2-4d39-bb01-b00fcab1903c","Type":"ContainerDied","Data":"b99e226ad4003aad7f6afff3420d66523de143e682b67251aca83b534e45b52d"}
Sep 11 11:48:27.252136 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:27.252097    2575 scope.go:117] "RemoveContainer" containerID="97c4b2afbb6bb124d9ce8d8e6b4cfd054c970c2ad529a2495e9c5778a8226d63"
Sep 11 11:48:27.270991 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:27.270960    2575 scope.go:117] "RemoveContainer" containerID="97c4b2afbb6bb124d9ce8d8e6b4cfd054c970c2ad529a2495e9c5778a8226d63"
Sep 11 11:48:27.271281 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:48:27.271259    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97c4b2afbb6bb124d9ce8d8e6b4cfd054c970c2ad529a2495e9c5778a8226d63\": container with ID starting with 97c4b2afbb6bb124d9ce8d8e6b4cfd054c970c2ad529a2495e9c5778a8226d63 not found: ID does not exist" containerID="97c4b2afbb6bb124d9ce8d8e6b4cfd054c970c2ad529a2495e9c5778a8226d63"
Sep 11 11:48:27.271348 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:27.271291    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97c4b2afbb6bb124d9ce8d8e6b4cfd054c970c2ad529a2495e9c5778a8226d63"} err="failed to get container status \"97c4b2afbb6bb124d9ce8d8e6b4cfd054c970c2ad529a2495e9c5778a8226d63\": rpc error: code = NotFound desc = could not find container \"97c4b2afbb6bb124d9ce8d8e6b4cfd054c970c2ad529a2495e9c5778a8226d63\": container with ID starting with 97c4b2afbb6bb124d9ce8d8e6b4cfd054c970c2ad529a2495e9c5778a8226d63 not found: ID does not exist"
Sep 11 11:48:27.289494 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:27.289465    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"]
Sep 11 11:48:27.292602 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:27.292576    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bddd3ec1-764c699fcc-h2db9"]
Sep 11 11:48:27.336447 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:27.336412    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:27.815444 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:27.815398    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:27.903048 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:27.903008    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:28.003153 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:28.003113    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:28.081891 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:28.081812    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2be5fb11-70a2-4d39-bb01-b00fcab1903c" path="/var/lib/kubelet/pods/2be5fb11-70a2-4d39-bb01-b00fcab1903c/volumes"
Sep 11 11:48:28.326015 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:28.325977    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:28.522945 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:28.522900    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:30.969513 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:30.969480    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:31.272556 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:31.272499    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:31.376716 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:31.376680    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:31.760584 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:31.760553    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"]
Sep 11 11:48:31.761070 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:31.761052    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2be5fb11-70a2-4d39-bb01-b00fcab1903c" containerName="maas-api"
Sep 11 11:48:31.761148 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:31.761072    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2be5fb11-70a2-4d39-bb01-b00fcab1903c" containerName="maas-api"
Sep 11 11:48:31.761200 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:31.761157    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2be5fb11-70a2-4d39-bb01-b00fcab1903c" containerName="maas-api"
Sep 11 11:48:31.764568 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:31.764549    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"
Sep 11 11:48:31.767098 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:31.767075    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-5fdde1a2-serving-cert\""
Sep 11 11:48:31.777284 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:31.777262    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"]
Sep 11 11:48:31.819108 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:31.819061    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:31.897327 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:31.897294    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/306d5dc1-d1f0-493f-9f19-288224eafa83-openshift-service-ca\") pod \"maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6\" (UID: \"306d5dc1-d1f0-493f-9f19-288224eafa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"
Sep 11 11:48:31.897493 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:31.897372    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/306d5dc1-d1f0-493f-9f19-288224eafa83-metrics-certs\") pod \"maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6\" (UID: \"306d5dc1-d1f0-493f-9f19-288224eafa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"
Sep 11 11:48:31.897493 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:31.897403    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/306d5dc1-d1f0-493f-9f19-288224eafa83-maas-api-tls\") pod \"maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6\" (UID: \"306d5dc1-d1f0-493f-9f19-288224eafa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"
Sep 11 11:48:31.897493 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:31.897417    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmh5z\" (UniqueName: \"kubernetes.io/projected/306d5dc1-d1f0-493f-9f19-288224eafa83-kube-api-access-bmh5z\") pod \"maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6\" (UID: \"306d5dc1-d1f0-493f-9f19-288224eafa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"
Sep 11 11:48:31.921816 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:31.921786    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:31.998996 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:31.998967    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/306d5dc1-d1f0-493f-9f19-288224eafa83-metrics-certs\") pod \"maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6\" (UID: \"306d5dc1-d1f0-493f-9f19-288224eafa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"
Sep 11 11:48:31.999444 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:31.999006    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/306d5dc1-d1f0-493f-9f19-288224eafa83-maas-api-tls\") pod \"maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6\" (UID: \"306d5dc1-d1f0-493f-9f19-288224eafa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"
Sep 11 11:48:31.999444 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:31.999034    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmh5z\" (UniqueName: \"kubernetes.io/projected/306d5dc1-d1f0-493f-9f19-288224eafa83-kube-api-access-bmh5z\") pod \"maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6\" (UID: \"306d5dc1-d1f0-493f-9f19-288224eafa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"
Sep 11 11:48:31.999444 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:31.999106    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/306d5dc1-d1f0-493f-9f19-288224eafa83-openshift-service-ca\") pod \"maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6\" (UID: \"306d5dc1-d1f0-493f-9f19-288224eafa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"
Sep 11 11:48:31.999444 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:48:31.999144    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-serving-cert: secret "maas-api-e2e-migrate-5fdde1a2-serving-cert" not found
Sep 11 11:48:31.999444 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:48:31.999216    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/306d5dc1-d1f0-493f-9f19-288224eafa83-maas-api-tls podName:306d5dc1-d1f0-493f-9f19-288224eafa83 nodeName:}" failed. No retries permitted until 2026-09-11 11:48:32.499195176 +0000 UTC m=+852.919801181 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/306d5dc1-d1f0-493f-9f19-288224eafa83-maas-api-tls") pod "maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6" (UID: "306d5dc1-d1f0-493f-9f19-288224eafa83") : secret "maas-api-e2e-migrate-5fdde1a2-serving-cert" not found
Sep 11 11:48:31.999810 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:31.999792    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/306d5dc1-d1f0-493f-9f19-288224eafa83-openshift-service-ca\") pod \"maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6\" (UID: \"306d5dc1-d1f0-493f-9f19-288224eafa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"
Sep 11 11:48:32.001501 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:32.001487    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/306d5dc1-d1f0-493f-9f19-288224eafa83-metrics-certs\") pod \"maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6\" (UID: \"306d5dc1-d1f0-493f-9f19-288224eafa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"
Sep 11 11:48:32.008011 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:32.007985    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmh5z\" (UniqueName: \"kubernetes.io/projected/306d5dc1-d1f0-493f-9f19-288224eafa83-kube-api-access-bmh5z\") pod \"maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6\" (UID: \"306d5dc1-d1f0-493f-9f19-288224eafa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"
Sep 11 11:48:32.290455 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:32.290424    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:32.397856 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:32.397821    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:32.504454 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:32.504415    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/306d5dc1-d1f0-493f-9f19-288224eafa83-maas-api-tls\") pod \"maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6\" (UID: \"306d5dc1-d1f0-493f-9f19-288224eafa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"
Sep 11 11:48:32.506888 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:32.506864    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/306d5dc1-d1f0-493f-9f19-288224eafa83-maas-api-tls\") pod \"maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6\" (UID: \"306d5dc1-d1f0-493f-9f19-288224eafa83\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"
Sep 11 11:48:32.676403 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:32.676326    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"
Sep 11 11:48:32.869850 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:32.869819    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:32.956187 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:32.956162    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"]
Sep 11 11:48:32.958085 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:48:32.958059    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod306d5dc1_d1f0_493f_9f19_288224eafa83.slice/crio-50fd6c8952569e39354630a1acc6516cc8f80082a55a99e2c8c3809fb9395084 WatchSource:0}: Error finding container 50fd6c8952569e39354630a1acc6516cc8f80082a55a99e2c8c3809fb9395084: Status 404 returned error can't find the container with id 50fd6c8952569e39354630a1acc6516cc8f80082a55a99e2c8c3809fb9395084
Sep 11 11:48:33.283248 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:33.283217    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6" event={"ID":"306d5dc1-d1f0-493f-9f19-288224eafa83","Type":"ContainerStarted","Data":"50fd6c8952569e39354630a1acc6516cc8f80082a55a99e2c8c3809fb9395084"}
Sep 11 11:48:33.700738 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:33.700711    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:34.289367 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:34.289331    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6" event={"ID":"306d5dc1-d1f0-493f-9f19-288224eafa83","Type":"ContainerStarted","Data":"d193c7d0a0c55270b4df3aa9bf70b7907614bd19cae95fa79a3f8bdec6d1254a"}
Sep 11 11:48:34.289854 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:34.289460    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"
Sep 11 11:48:34.309018 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:34.308961    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6" podStartSLOduration=2.586184628 podStartE2EDuration="3.308941748s" totalImagesPullingTime="722.75712ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:48:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:48:34.308214897 +0000 UTC m=+854.728820937" watchObservedRunningTime="2026-09-11 11:48:34.308941748 +0000 UTC m=+854.729547775"
Sep 11 11:48:36.387694 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:36.387661    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:40.300403 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:40.300370    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"
Sep 11 11:48:59.731970 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.731936    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"]
Sep 11 11:48:59.736194 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.736173    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:48:59.740290 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.740268    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-5a5ab7\"/\"autores-model-5a5ab7-kserve-self-signed-certs\""
Sep 11 11:48:59.740290 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.740286    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-5a5ab7\"/\"default-dockercfg-j77fb\""
Sep 11 11:48:59.746914 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.746893    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-5a5ab7\"/\"openshift-service-ca.crt\""
Sep 11 11:48:59.747013 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.746998    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-5a5ab7\"/\"kube-root-ca.crt\""
Sep 11 11:48:59.757157 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.757140    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-tmp-dir\") pod \"autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") " pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:48:59.757238 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.757191    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-home\") pod \"autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") " pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:48:59.757238 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.757209    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4308f1c-09b4-4658-949c-9858d9791554-tls-certs\") pod \"autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") " pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:48:59.757315 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.757246    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbgnl\" (UniqueName: \"kubernetes.io/projected/a4308f1c-09b4-4658-949c-9858d9791554-kube-api-access-fbgnl\") pod \"autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") " pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:48:59.757315 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.757279    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-model-cache\") pod \"autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") " pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:48:59.757315 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.757294    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-dshm\") pod \"autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") " pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:48:59.769595 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.769577    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"]
Sep 11 11:48:59.809190 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.809163    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:48:59.858357 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.858333    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-tmp-dir\") pod \"autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") " pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:48:59.858466 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.858402    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-home\") pod \"autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") " pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:48:59.858466 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.858423    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4308f1c-09b4-4658-949c-9858d9791554-tls-certs\") pod \"autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") " pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:48:59.858466 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.858452    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbgnl\" (UniqueName: \"kubernetes.io/projected/a4308f1c-09b4-4658-949c-9858d9791554-kube-api-access-fbgnl\") pod \"autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") " pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:48:59.858621 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.858492    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-model-cache\") pod \"autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") " pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:48:59.858621 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.858510    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-dshm\") pod \"autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") " pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:48:59.858798 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.858777    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-tmp-dir\") pod \"autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") " pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:48:59.858858 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.858836    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-home\") pod \"autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") " pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:48:59.858909 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.858867    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-model-cache\") pod \"autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") " pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:48:59.860903 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.860882    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-dshm\") pod \"autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") " pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:48:59.861048 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.861030    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4308f1c-09b4-4658-949c-9858d9791554-tls-certs\") pod \"autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") " pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:48:59.869294 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:48:59.869268    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbgnl\" (UniqueName: \"kubernetes.io/projected/a4308f1c-09b4-4658-949c-9858d9791554-kube-api-access-fbgnl\") pod \"autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") " pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:49:00.045800 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:00.045768    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:49:00.319817 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:00.319792    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"]
Sep 11 11:49:00.321485 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:49:00.321460    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4308f1c_09b4_4658_949c_9858d9791554.slice/crio-c4770768ab567237fc6cfcfca819b9b097d4c7b73db8b25f0c2082136dce9ad6 WatchSource:0}: Error finding container c4770768ab567237fc6cfcfca819b9b097d4c7b73db8b25f0c2082136dce9ad6: Status 404 returned error can't find the container with id c4770768ab567237fc6cfcfca819b9b097d4c7b73db8b25f0c2082136dce9ad6
Sep 11 11:49:00.424172 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:00.424143    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq" event={"ID":"a4308f1c-09b4-4658-949c-9858d9791554","Type":"ContainerStarted","Data":"d459854cc75ef9938627703303fa575cda966c9183ee002361765d465e67f781"}
Sep 11 11:49:00.424304 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:00.424189    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq" event={"ID":"a4308f1c-09b4-4658-949c-9858d9791554","Type":"ContainerStarted","Data":"c4770768ab567237fc6cfcfca819b9b097d4c7b73db8b25f0c2082136dce9ad6"}
Sep 11 11:49:00.445089 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:00.445045    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq" podStartSLOduration=1.445029725 podStartE2EDuration="1.445029725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:48:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:49:00.444249211 +0000 UTC m=+880.864855239" watchObservedRunningTime="2026-09-11 11:49:00.445029725 +0000 UTC m=+880.865635753"
Sep 11 11:49:02.730819 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:02.730781    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"]
Sep 11 11:49:02.731201 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:02.731024    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6" podUID="306d5dc1-d1f0-493f-9f19-288224eafa83" containerName="maas-api" containerID="cri-o://d193c7d0a0c55270b4df3aa9bf70b7907614bd19cae95fa79a3f8bdec6d1254a" gracePeriod=30
Sep 11 11:49:02.856574 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:02.856505    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:02.954092 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:02.954057    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:02.983998 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:02.983946    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"
Sep 11 11:49:03.056098 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.056061    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:03.089266 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.088821    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/306d5dc1-d1f0-493f-9f19-288224eafa83-metrics-certs\") pod \"306d5dc1-d1f0-493f-9f19-288224eafa83\" (UID: \"306d5dc1-d1f0-493f-9f19-288224eafa83\") "
Sep 11 11:49:03.089266 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.088916    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/306d5dc1-d1f0-493f-9f19-288224eafa83-maas-api-tls\") pod \"306d5dc1-d1f0-493f-9f19-288224eafa83\" (UID: \"306d5dc1-d1f0-493f-9f19-288224eafa83\") "
Sep 11 11:49:03.089266 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.088993    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/306d5dc1-d1f0-493f-9f19-288224eafa83-openshift-service-ca\") pod \"306d5dc1-d1f0-493f-9f19-288224eafa83\" (UID: \"306d5dc1-d1f0-493f-9f19-288224eafa83\") "
Sep 11 11:49:03.089266 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.089075    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmh5z\" (UniqueName: \"kubernetes.io/projected/306d5dc1-d1f0-493f-9f19-288224eafa83-kube-api-access-bmh5z\") pod \"306d5dc1-d1f0-493f-9f19-288224eafa83\" (UID: \"306d5dc1-d1f0-493f-9f19-288224eafa83\") "
Sep 11 11:49:03.090018 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.089990    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/306d5dc1-d1f0-493f-9f19-288224eafa83-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "306d5dc1-d1f0-493f-9f19-288224eafa83" (UID: "306d5dc1-d1f0-493f-9f19-288224eafa83"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:49:03.091462 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.091433    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/306d5dc1-d1f0-493f-9f19-288224eafa83-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "306d5dc1-d1f0-493f-9f19-288224eafa83" (UID: "306d5dc1-d1f0-493f-9f19-288224eafa83"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:49:03.092023 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.091999    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/306d5dc1-d1f0-493f-9f19-288224eafa83-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "306d5dc1-d1f0-493f-9f19-288224eafa83" (UID: "306d5dc1-d1f0-493f-9f19-288224eafa83"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:49:03.092023 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.092014    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/306d5dc1-d1f0-493f-9f19-288224eafa83-kube-api-access-bmh5z" (OuterVolumeSpecName: "kube-api-access-bmh5z") pod "306d5dc1-d1f0-493f-9f19-288224eafa83" (UID: "306d5dc1-d1f0-493f-9f19-288224eafa83"). InnerVolumeSpecName "kube-api-access-bmh5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:49:03.155427 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.155389    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:03.190759 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.190733    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/306d5dc1-d1f0-493f-9f19-288224eafa83-metrics-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:49:03.190759 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.190755    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/306d5dc1-d1f0-493f-9f19-288224eafa83-maas-api-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:49:03.190932 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.190766    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/306d5dc1-d1f0-493f-9f19-288224eafa83-openshift-service-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:49:03.190932 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.190776    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmh5z\" (UniqueName: \"kubernetes.io/projected/306d5dc1-d1f0-493f-9f19-288224eafa83-kube-api-access-bmh5z\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:49:03.439990 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.439956    2575 generic.go:356] "Generic (PLEG): container finished" podID="306d5dc1-d1f0-493f-9f19-288224eafa83" containerID="d193c7d0a0c55270b4df3aa9bf70b7907614bd19cae95fa79a3f8bdec6d1254a" exitCode=0
Sep 11 11:49:03.440159 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.440003    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"
Sep 11 11:49:03.440159 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.440038    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6" event={"ID":"306d5dc1-d1f0-493f-9f19-288224eafa83","Type":"ContainerDied","Data":"d193c7d0a0c55270b4df3aa9bf70b7907614bd19cae95fa79a3f8bdec6d1254a"}
Sep 11 11:49:03.440159 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.440092    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6" event={"ID":"306d5dc1-d1f0-493f-9f19-288224eafa83","Type":"ContainerDied","Data":"50fd6c8952569e39354630a1acc6516cc8f80082a55a99e2c8c3809fb9395084"}
Sep 11 11:49:03.440159 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.440113    2575 scope.go:117] "RemoveContainer" containerID="d193c7d0a0c55270b4df3aa9bf70b7907614bd19cae95fa79a3f8bdec6d1254a"
Sep 11 11:49:03.451539 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.451505    2575 scope.go:117] "RemoveContainer" containerID="d193c7d0a0c55270b4df3aa9bf70b7907614bd19cae95fa79a3f8bdec6d1254a"
Sep 11 11:49:03.452031 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:49:03.452010    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d193c7d0a0c55270b4df3aa9bf70b7907614bd19cae95fa79a3f8bdec6d1254a\": container with ID starting with d193c7d0a0c55270b4df3aa9bf70b7907614bd19cae95fa79a3f8bdec6d1254a not found: ID does not exist" containerID="d193c7d0a0c55270b4df3aa9bf70b7907614bd19cae95fa79a3f8bdec6d1254a"
Sep 11 11:49:03.452123 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.452037    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d193c7d0a0c55270b4df3aa9bf70b7907614bd19cae95fa79a3f8bdec6d1254a"} err="failed to get container status \"d193c7d0a0c55270b4df3aa9bf70b7907614bd19cae95fa79a3f8bdec6d1254a\": rpc error: code = NotFound desc = could not find container \"d193c7d0a0c55270b4df3aa9bf70b7907614bd19cae95fa79a3f8bdec6d1254a\": container with ID starting with d193c7d0a0c55270b4df3aa9bf70b7907614bd19cae95fa79a3f8bdec6d1254a not found: ID does not exist"
Sep 11 11:49:03.461381 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.461360    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"]
Sep 11 11:49:03.465411 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.465392    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-5fdde1a2-7c977f4774-67gq6"]
Sep 11 11:49:03.614889 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.614859    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:03.928727 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:03.928698    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:04.083217 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:04.083183    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="306d5dc1-d1f0-493f-9f19-288224eafa83" path="/var/lib/kubelet/pods/306d5dc1-d1f0-493f-9f19-288224eafa83/volumes"
Sep 11 11:49:04.447806 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:04.447779    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:08.934269 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:08.934233    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:09.034100 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:09.034068    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:10.046657 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.046594    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:49:10.047102 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.046676    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:49:10.047866 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.047841    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"]
Sep 11 11:49:10.048405 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.048384    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="306d5dc1-d1f0-493f-9f19-288224eafa83" containerName="maas-api"
Sep 11 11:49:10.048405 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.048403    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="306d5dc1-d1f0-493f-9f19-288224eafa83" containerName="maas-api"
Sep 11 11:49:10.048592 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.048509    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="306d5dc1-d1f0-493f-9f19-288224eafa83" containerName="maas-api"
Sep 11 11:49:10.053362 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.053339    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"
Sep 11 11:49:10.055964 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.055944    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-8ede8f25-serving-cert\""
Sep 11 11:49:10.060749 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.060734    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:49:10.070786 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.070541    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"]
Sep 11 11:49:10.155764 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.155732    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf5gq\" (UniqueName: \"kubernetes.io/projected/59845617-b24a-4dfb-9520-1a6b33c21e75-kube-api-access-jf5gq\") pod \"maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp\" (UID: \"59845617-b24a-4dfb-9520-1a6b33c21e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"
Sep 11 11:49:10.156051 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.156019    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59845617-b24a-4dfb-9520-1a6b33c21e75-openshift-service-ca\") pod \"maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp\" (UID: \"59845617-b24a-4dfb-9520-1a6b33c21e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"
Sep 11 11:49:10.156186 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.156162    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59845617-b24a-4dfb-9520-1a6b33c21e75-metrics-certs\") pod \"maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp\" (UID: \"59845617-b24a-4dfb-9520-1a6b33c21e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"
Sep 11 11:49:10.156275 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.156262    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59845617-b24a-4dfb-9520-1a6b33c21e75-maas-api-tls\") pod \"maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp\" (UID: \"59845617-b24a-4dfb-9520-1a6b33c21e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"
Sep 11 11:49:10.161875 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.161853    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:10.256728 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.256699    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59845617-b24a-4dfb-9520-1a6b33c21e75-maas-api-tls\") pod \"maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp\" (UID: \"59845617-b24a-4dfb-9520-1a6b33c21e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"
Sep 11 11:49:10.256892 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.256736    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf5gq\" (UniqueName: \"kubernetes.io/projected/59845617-b24a-4dfb-9520-1a6b33c21e75-kube-api-access-jf5gq\") pod \"maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp\" (UID: \"59845617-b24a-4dfb-9520-1a6b33c21e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"
Sep 11 11:49:10.256892 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.256800    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59845617-b24a-4dfb-9520-1a6b33c21e75-openshift-service-ca\") pod \"maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp\" (UID: \"59845617-b24a-4dfb-9520-1a6b33c21e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"
Sep 11 11:49:10.256892 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.256828    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59845617-b24a-4dfb-9520-1a6b33c21e75-metrics-certs\") pod \"maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp\" (UID: \"59845617-b24a-4dfb-9520-1a6b33c21e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"
Sep 11 11:49:10.257583 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.257558    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59845617-b24a-4dfb-9520-1a6b33c21e75-openshift-service-ca\") pod \"maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp\" (UID: \"59845617-b24a-4dfb-9520-1a6b33c21e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"
Sep 11 11:49:10.259476 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.259454    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59845617-b24a-4dfb-9520-1a6b33c21e75-maas-api-tls\") pod \"maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp\" (UID: \"59845617-b24a-4dfb-9520-1a6b33c21e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"
Sep 11 11:49:10.259677 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.259659    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59845617-b24a-4dfb-9520-1a6b33c21e75-metrics-certs\") pod \"maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp\" (UID: \"59845617-b24a-4dfb-9520-1a6b33c21e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"
Sep 11 11:49:10.263134 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.263101    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:10.266187 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.266162    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf5gq\" (UniqueName: \"kubernetes.io/projected/59845617-b24a-4dfb-9520-1a6b33c21e75-kube-api-access-jf5gq\") pod \"maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp\" (UID: \"59845617-b24a-4dfb-9520-1a6b33c21e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"
Sep 11 11:49:10.363082 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.362999    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:10.392055 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.392029    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"
Sep 11 11:49:10.489947 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.489921    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:49:10.644691 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.644585    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:10.670471 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.670445    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"]
Sep 11 11:49:10.671827 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:49:10.671800    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59845617_b24a_4dfb_9520_1a6b33c21e75.slice/crio-183033d58b110938bdf5281e398d0855e5792ff40c36aebef44ac03e0c726e2d WatchSource:0}: Error finding container 183033d58b110938bdf5281e398d0855e5792ff40c36aebef44ac03e0c726e2d: Status 404 returned error can't find the container with id 183033d58b110938bdf5281e398d0855e5792ff40c36aebef44ac03e0c726e2d
Sep 11 11:49:10.727106 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:10.727078    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:11.486468 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:11.486433    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp" event={"ID":"59845617-b24a-4dfb-9520-1a6b33c21e75","Type":"ContainerStarted","Data":"5fb4bb0f7f0e827a3ed8af10332c87dcec9fbc2b36e1dcdfad29c42a3db9efd3"}
Sep 11 11:49:11.486886 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:11.486497    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp" event={"ID":"59845617-b24a-4dfb-9520-1a6b33c21e75","Type":"ContainerStarted","Data":"183033d58b110938bdf5281e398d0855e5792ff40c36aebef44ac03e0c726e2d"}
Sep 11 11:49:11.486886 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:11.486534    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"
Sep 11 11:49:17.497177 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:17.497147    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"
Sep 11 11:49:17.518620 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:17.518565    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp" podStartSLOduration=7.086392263 podStartE2EDuration="7.518544226s" totalImagesPullingTime="432.151963ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:49:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:49:11.505153955 +0000 UTC m=+891.925759983" watchObservedRunningTime="2026-09-11 11:49:17.518544226 +0000 UTC m=+897.939150254"
Sep 11 11:49:35.378210 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:35.378167    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:37.127894 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:37.127865    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:40.078731 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:40.078694    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:40.140196 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:40.140165    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:40.805469 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:40.805436    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"]
Sep 11 11:49:40.805736 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:40.805694    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp" podUID="59845617-b24a-4dfb-9520-1a6b33c21e75" containerName="maas-api" containerID="cri-o://5fb4bb0f7f0e827a3ed8af10332c87dcec9fbc2b36e1dcdfad29c42a3db9efd3" gracePeriod=30
Sep 11 11:49:40.829744 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:40.829655    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:40.930487 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:40.930442    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:41.030653 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.030604    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:41.062100 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.062046    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"
Sep 11 11:49:41.132821 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.132767    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:41.152461 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.152441    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jf5gq\" (UniqueName: \"kubernetes.io/projected/59845617-b24a-4dfb-9520-1a6b33c21e75-kube-api-access-jf5gq\") pod \"59845617-b24a-4dfb-9520-1a6b33c21e75\" (UID: \"59845617-b24a-4dfb-9520-1a6b33c21e75\") "
Sep 11 11:49:41.152588 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.152487    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59845617-b24a-4dfb-9520-1a6b33c21e75-metrics-certs\") pod \"59845617-b24a-4dfb-9520-1a6b33c21e75\" (UID: \"59845617-b24a-4dfb-9520-1a6b33c21e75\") "
Sep 11 11:49:41.152588 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.152573    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59845617-b24a-4dfb-9520-1a6b33c21e75-maas-api-tls\") pod \"59845617-b24a-4dfb-9520-1a6b33c21e75\" (UID: \"59845617-b24a-4dfb-9520-1a6b33c21e75\") "
Sep 11 11:49:41.152710 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.152615    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59845617-b24a-4dfb-9520-1a6b33c21e75-openshift-service-ca\") pod \"59845617-b24a-4dfb-9520-1a6b33c21e75\" (UID: \"59845617-b24a-4dfb-9520-1a6b33c21e75\") "
Sep 11 11:49:41.153026 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.152997    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59845617-b24a-4dfb-9520-1a6b33c21e75-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "59845617-b24a-4dfb-9520-1a6b33c21e75" (UID: "59845617-b24a-4dfb-9520-1a6b33c21e75"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:49:41.154709 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.154679    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59845617-b24a-4dfb-9520-1a6b33c21e75-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "59845617-b24a-4dfb-9520-1a6b33c21e75" (UID: "59845617-b24a-4dfb-9520-1a6b33c21e75"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:49:41.154801 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.154730    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59845617-b24a-4dfb-9520-1a6b33c21e75-kube-api-access-jf5gq" (OuterVolumeSpecName: "kube-api-access-jf5gq") pod "59845617-b24a-4dfb-9520-1a6b33c21e75" (UID: "59845617-b24a-4dfb-9520-1a6b33c21e75"). InnerVolumeSpecName "kube-api-access-jf5gq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:49:41.154801 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.154758    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59845617-b24a-4dfb-9520-1a6b33c21e75-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "59845617-b24a-4dfb-9520-1a6b33c21e75" (UID: "59845617-b24a-4dfb-9520-1a6b33c21e75"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:49:41.253924 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.253895    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59845617-b24a-4dfb-9520-1a6b33c21e75-maas-api-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:49:41.253924 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.253918    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59845617-b24a-4dfb-9520-1a6b33c21e75-openshift-service-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:49:41.253924 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.253928    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jf5gq\" (UniqueName: \"kubernetes.io/projected/59845617-b24a-4dfb-9520-1a6b33c21e75-kube-api-access-jf5gq\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:49:41.254129 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.253939    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59845617-b24a-4dfb-9520-1a6b33c21e75-metrics-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:49:41.639987 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.639953    2575 generic.go:356] "Generic (PLEG): container finished" podID="59845617-b24a-4dfb-9520-1a6b33c21e75" containerID="5fb4bb0f7f0e827a3ed8af10332c87dcec9fbc2b36e1dcdfad29c42a3db9efd3" exitCode=0
Sep 11 11:49:41.640159 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.639998    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"
Sep 11 11:49:41.640159 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.640041    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp" event={"ID":"59845617-b24a-4dfb-9520-1a6b33c21e75","Type":"ContainerDied","Data":"5fb4bb0f7f0e827a3ed8af10332c87dcec9fbc2b36e1dcdfad29c42a3db9efd3"}
Sep 11 11:49:41.640159 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.640095    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp" event={"ID":"59845617-b24a-4dfb-9520-1a6b33c21e75","Type":"ContainerDied","Data":"183033d58b110938bdf5281e398d0855e5792ff40c36aebef44ac03e0c726e2d"}
Sep 11 11:49:41.640159 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.640123    2575 scope.go:117] "RemoveContainer" containerID="5fb4bb0f7f0e827a3ed8af10332c87dcec9fbc2b36e1dcdfad29c42a3db9efd3"
Sep 11 11:49:41.649713 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.649698    2575 scope.go:117] "RemoveContainer" containerID="5fb4bb0f7f0e827a3ed8af10332c87dcec9fbc2b36e1dcdfad29c42a3db9efd3"
Sep 11 11:49:41.649960 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:49:41.649945    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fb4bb0f7f0e827a3ed8af10332c87dcec9fbc2b36e1dcdfad29c42a3db9efd3\": container with ID starting with 5fb4bb0f7f0e827a3ed8af10332c87dcec9fbc2b36e1dcdfad29c42a3db9efd3 not found: ID does not exist" containerID="5fb4bb0f7f0e827a3ed8af10332c87dcec9fbc2b36e1dcdfad29c42a3db9efd3"
Sep 11 11:49:41.650005 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.649968    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fb4bb0f7f0e827a3ed8af10332c87dcec9fbc2b36e1dcdfad29c42a3db9efd3"} err="failed to get container status \"5fb4bb0f7f0e827a3ed8af10332c87dcec9fbc2b36e1dcdfad29c42a3db9efd3\": rpc error: code = NotFound desc = could not find container \"5fb4bb0f7f0e827a3ed8af10332c87dcec9fbc2b36e1dcdfad29c42a3db9efd3\": container with ID starting with 5fb4bb0f7f0e827a3ed8af10332c87dcec9fbc2b36e1dcdfad29c42a3db9efd3 not found: ID does not exist"
Sep 11 11:49:41.662967 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.662949    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"]
Sep 11 11:49:41.669019 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:41.669001    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8ede8f25-76cfd94cf4-s58jp"]
Sep 11 11:49:42.082099 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:42.082069    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59845617-b24a-4dfb-9520-1a6b33c21e75" path="/var/lib/kubelet/pods/59845617-b24a-4dfb-9520-1a6b33c21e75/volumes"
Sep 11 11:49:42.285156 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:42.285113    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:42.753261 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:42.753227    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:43.065405 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:43.065322    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:43.322218 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:43.322122    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:44.723025 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.722997    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"]
Sep 11 11:49:44.723461 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.723419    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59845617-b24a-4dfb-9520-1a6b33c21e75" containerName="maas-api"
Sep 11 11:49:44.723461 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.723429    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="59845617-b24a-4dfb-9520-1a6b33c21e75" containerName="maas-api"
Sep 11 11:49:44.723580 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.723497    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="59845617-b24a-4dfb-9520-1a6b33c21e75" containerName="maas-api"
Sep 11 11:49:44.728064 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.728042    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:49:44.730556 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.730535    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-31a432b7-kserve-self-signed-certs\""
Sep 11 11:49:44.738300 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.738279    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"]
Sep 11 11:49:44.742744 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.742719    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:44.784510 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.784482    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-model-cache\") pod \"e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") " pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:49:44.784675 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.784528    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mt6d\" (UniqueName: \"kubernetes.io/projected/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-kube-api-access-4mt6d\") pod \"e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") " pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:49:44.784675 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.784577    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-dshm\") pod \"e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") " pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:49:44.784675 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.784599    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-home\") pod \"e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") " pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:49:44.784675 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.784625    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-tmp-dir\") pod \"e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") " pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:49:44.784846 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.784712    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-tls-certs\") pod \"e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") " pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:49:44.885565 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.885531    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-tls-certs\") pod \"e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") " pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:49:44.885565 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.885564    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-model-cache\") pod \"e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") " pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:49:44.885774 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.885599    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mt6d\" (UniqueName: \"kubernetes.io/projected/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-kube-api-access-4mt6d\") pod \"e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") " pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:49:44.885821 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.885804    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-dshm\") pod \"e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") " pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:49:44.885875 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.885850    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-home\") pod \"e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") " pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:49:44.885932 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.885874    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-tmp-dir\") pod \"e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") " pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:49:44.886054 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.886029    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-model-cache\") pod \"e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") " pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:49:44.886322 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.886286    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-home\") pod \"e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") " pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:49:44.886453 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.886293    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-tmp-dir\") pod \"e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") " pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:49:44.888387 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.888361    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-tls-certs\") pod \"e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") " pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:49:44.888505 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.888411    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-dshm\") pod \"e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") " pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:49:44.893846 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:44.893826    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mt6d\" (UniqueName: \"kubernetes.io/projected/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-kube-api-access-4mt6d\") pod \"e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") " pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:49:45.039611 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:45.039577    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:49:45.045766 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:45.045739    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:45.147038 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:45.147009    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:45.244072 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:45.244042    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:45.316351 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:45.316325    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"]
Sep 11 11:49:45.320599 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:49:45.318261    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd673fd55_264f_4c76_8c78_c2af4a4ca4ad.slice/crio-f710b038869a25160bf7eddfbc99dcfd56737582cb8eb37e4d8324ca46c0a310 WatchSource:0}: Error finding container f710b038869a25160bf7eddfbc99dcfd56737582cb8eb37e4d8324ca46c0a310: Status 404 returned error can't find the container with id f710b038869a25160bf7eddfbc99dcfd56737582cb8eb37e4d8324ca46c0a310
Sep 11 11:49:45.675155 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:45.675055    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"]
Sep 11 11:49:45.675318 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:45.675296    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9" podUID="fa73d8ab-d829-47d9-84d2-f6fd20c954af" containerName="maas-api" containerID="cri-o://9e4904525afc89e02789e9e6c9d1f3ebf6af325e4edf9ddbc394a865985922ba" gracePeriod=30
Sep 11 11:49:45.680592 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:45.680488    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s" event={"ID":"d673fd55-264f-4c76-8c78-c2af4a4ca4ad","Type":"ContainerStarted","Data":"909bb000dcbba4f1a6c19a5f2d5bc58acc1a6b8cb29038a4afcd196c1663085e"}
Sep 11 11:49:45.680592 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:45.680528    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s" event={"ID":"d673fd55-264f-4c76-8c78-c2af4a4ca4ad","Type":"ContainerStarted","Data":"f710b038869a25160bf7eddfbc99dcfd56737582cb8eb37e4d8324ca46c0a310"}
Sep 11 11:49:45.709393 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:45.709347    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:45.718535 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:45.718484    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s" podStartSLOduration=1.718463912 podStartE2EDuration="1.718463912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:49:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:49:45.716852771 +0000 UTC m=+926.137458797" watchObservedRunningTime="2026-09-11 11:49:45.718463912 +0000 UTC m=+926.139069941"
Sep 11 11:49:45.810009 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:45.809798    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:45.911957 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:45.911927    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:45.931604 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:45.931542    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"
Sep 11 11:49:45.998786 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:45.998759    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa73d8ab-d829-47d9-84d2-f6fd20c954af-maas-api-tls\") pod \"fa73d8ab-d829-47d9-84d2-f6fd20c954af\" (UID: \"fa73d8ab-d829-47d9-84d2-f6fd20c954af\") "
Sep 11 11:49:45.998905 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:45.998810    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa73d8ab-d829-47d9-84d2-f6fd20c954af-metrics-certs\") pod \"fa73d8ab-d829-47d9-84d2-f6fd20c954af\" (UID: \"fa73d8ab-d829-47d9-84d2-f6fd20c954af\") "
Sep 11 11:49:45.998905 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:45.998832    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa73d8ab-d829-47d9-84d2-f6fd20c954af-openshift-service-ca\") pod \"fa73d8ab-d829-47d9-84d2-f6fd20c954af\" (UID: \"fa73d8ab-d829-47d9-84d2-f6fd20c954af\") "
Sep 11 11:49:45.998905 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:45.998894    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmzdd\" (UniqueName: \"kubernetes.io/projected/fa73d8ab-d829-47d9-84d2-f6fd20c954af-kube-api-access-lmzdd\") pod \"fa73d8ab-d829-47d9-84d2-f6fd20c954af\" (UID: \"fa73d8ab-d829-47d9-84d2-f6fd20c954af\") "
Sep 11 11:49:45.999228 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:45.999200    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa73d8ab-d829-47d9-84d2-f6fd20c954af-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fa73d8ab-d829-47d9-84d2-f6fd20c954af" (UID: "fa73d8ab-d829-47d9-84d2-f6fd20c954af"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:49:46.001048 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.001022    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa73d8ab-d829-47d9-84d2-f6fd20c954af-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fa73d8ab-d829-47d9-84d2-f6fd20c954af" (UID: "fa73d8ab-d829-47d9-84d2-f6fd20c954af"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:49:46.001048 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.001033    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa73d8ab-d829-47d9-84d2-f6fd20c954af-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fa73d8ab-d829-47d9-84d2-f6fd20c954af" (UID: "fa73d8ab-d829-47d9-84d2-f6fd20c954af"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:49:46.001168 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.001049    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa73d8ab-d829-47d9-84d2-f6fd20c954af-kube-api-access-lmzdd" (OuterVolumeSpecName: "kube-api-access-lmzdd") pod "fa73d8ab-d829-47d9-84d2-f6fd20c954af" (UID: "fa73d8ab-d829-47d9-84d2-f6fd20c954af"). InnerVolumeSpecName "kube-api-access-lmzdd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:49:46.010740 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.010708    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:46.100075 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.100053    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa73d8ab-d829-47d9-84d2-f6fd20c954af-maas-api-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:49:46.100075 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.100074    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa73d8ab-d829-47d9-84d2-f6fd20c954af-metrics-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:49:46.100227 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.100084    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa73d8ab-d829-47d9-84d2-f6fd20c954af-openshift-service-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:49:46.100227 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.100093    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmzdd\" (UniqueName: \"kubernetes.io/projected/fa73d8ab-d829-47d9-84d2-f6fd20c954af-kube-api-access-lmzdd\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:49:46.299276 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.299238    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"]
Sep 11 11:49:46.299983 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.299956    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa73d8ab-d829-47d9-84d2-f6fd20c954af" containerName="maas-api"
Sep 11 11:49:46.299983 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.299981    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa73d8ab-d829-47d9-84d2-f6fd20c954af" containerName="maas-api"
Sep 11 11:49:46.300178 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.300093    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa73d8ab-d829-47d9-84d2-f6fd20c954af" containerName="maas-api"
Sep 11 11:49:46.303935 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.303912    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"
Sep 11 11:49:46.306313 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.306291    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-d507e5c9-serving-cert\""
Sep 11 11:49:46.313883 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.313862    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"]
Sep 11 11:49:46.362708 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.362684    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:46.403715 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.403686    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b84e57a-c0b0-4490-94fe-a02c6f746031-metrics-certs\") pod \"maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s\" (UID: \"9b84e57a-c0b0-4490-94fe-a02c6f746031\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"
Sep 11 11:49:46.403856 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.403723    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b84e57a-c0b0-4490-94fe-a02c6f746031-openshift-service-ca\") pod \"maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s\" (UID: \"9b84e57a-c0b0-4490-94fe-a02c6f746031\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"
Sep 11 11:49:46.403856 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.403760    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b84e57a-c0b0-4490-94fe-a02c6f746031-maas-api-tls\") pod \"maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s\" (UID: \"9b84e57a-c0b0-4490-94fe-a02c6f746031\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"
Sep 11 11:49:46.403856 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.403828    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sl75\" (UniqueName: \"kubernetes.io/projected/9b84e57a-c0b0-4490-94fe-a02c6f746031-kube-api-access-7sl75\") pod \"maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s\" (UID: \"9b84e57a-c0b0-4490-94fe-a02c6f746031\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"
Sep 11 11:49:46.462660 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.462609    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:46.504961 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.504929    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b84e57a-c0b0-4490-94fe-a02c6f746031-maas-api-tls\") pod \"maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s\" (UID: \"9b84e57a-c0b0-4490-94fe-a02c6f746031\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"
Sep 11 11:49:46.505138 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.504989    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sl75\" (UniqueName: \"kubernetes.io/projected/9b84e57a-c0b0-4490-94fe-a02c6f746031-kube-api-access-7sl75\") pod \"maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s\" (UID: \"9b84e57a-c0b0-4490-94fe-a02c6f746031\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"
Sep 11 11:49:46.505138 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.505065    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b84e57a-c0b0-4490-94fe-a02c6f746031-metrics-certs\") pod \"maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s\" (UID: \"9b84e57a-c0b0-4490-94fe-a02c6f746031\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"
Sep 11 11:49:46.505138 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.505091    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b84e57a-c0b0-4490-94fe-a02c6f746031-openshift-service-ca\") pod \"maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s\" (UID: \"9b84e57a-c0b0-4490-94fe-a02c6f746031\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"
Sep 11 11:49:46.505826 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.505769    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b84e57a-c0b0-4490-94fe-a02c6f746031-openshift-service-ca\") pod \"maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s\" (UID: \"9b84e57a-c0b0-4490-94fe-a02c6f746031\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"
Sep 11 11:49:46.507604 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.507584    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b84e57a-c0b0-4490-94fe-a02c6f746031-metrics-certs\") pod \"maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s\" (UID: \"9b84e57a-c0b0-4490-94fe-a02c6f746031\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"
Sep 11 11:49:46.507718 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.507681    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b84e57a-c0b0-4490-94fe-a02c6f746031-maas-api-tls\") pod \"maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s\" (UID: \"9b84e57a-c0b0-4490-94fe-a02c6f746031\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"
Sep 11 11:49:46.513038 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.513020    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sl75\" (UniqueName: \"kubernetes.io/projected/9b84e57a-c0b0-4490-94fe-a02c6f746031-kube-api-access-7sl75\") pod \"maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s\" (UID: \"9b84e57a-c0b0-4490-94fe-a02c6f746031\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"
Sep 11 11:49:46.561367 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.561264    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:46.616723 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.616687    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"
Sep 11 11:49:46.687842 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.687810    2575 generic.go:356] "Generic (PLEG): container finished" podID="fa73d8ab-d829-47d9-84d2-f6fd20c954af" containerID="9e4904525afc89e02789e9e6c9d1f3ebf6af325e4edf9ddbc394a865985922ba" exitCode=0
Sep 11 11:49:46.687982 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.687861    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"
Sep 11 11:49:46.687982 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.687892    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9" event={"ID":"fa73d8ab-d829-47d9-84d2-f6fd20c954af","Type":"ContainerDied","Data":"9e4904525afc89e02789e9e6c9d1f3ebf6af325e4edf9ddbc394a865985922ba"}
Sep 11 11:49:46.687982 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.687954    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9" event={"ID":"fa73d8ab-d829-47d9-84d2-f6fd20c954af","Type":"ContainerDied","Data":"64e8cd6bf33794b7cf2c4f846750832ab7098080ff7b7a3dab1608d519887d24"}
Sep 11 11:49:46.688130 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.687983    2575 scope.go:117] "RemoveContainer" containerID="9e4904525afc89e02789e9e6c9d1f3ebf6af325e4edf9ddbc394a865985922ba"
Sep 11 11:49:46.698886 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.698865    2575 scope.go:117] "RemoveContainer" containerID="9e4904525afc89e02789e9e6c9d1f3ebf6af325e4edf9ddbc394a865985922ba"
Sep 11 11:49:46.699186 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:49:46.699158    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e4904525afc89e02789e9e6c9d1f3ebf6af325e4edf9ddbc394a865985922ba\": container with ID starting with 9e4904525afc89e02789e9e6c9d1f3ebf6af325e4edf9ddbc394a865985922ba not found: ID does not exist" containerID="9e4904525afc89e02789e9e6c9d1f3ebf6af325e4edf9ddbc394a865985922ba"
Sep 11 11:49:46.699258 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.699188    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e4904525afc89e02789e9e6c9d1f3ebf6af325e4edf9ddbc394a865985922ba"} err="failed to get container status \"9e4904525afc89e02789e9e6c9d1f3ebf6af325e4edf9ddbc394a865985922ba\": rpc error: code = NotFound desc = could not find container \"9e4904525afc89e02789e9e6c9d1f3ebf6af325e4edf9ddbc394a865985922ba\": container with ID starting with 9e4904525afc89e02789e9e6c9d1f3ebf6af325e4edf9ddbc394a865985922ba not found: ID does not exist"
Sep 11 11:49:46.705503 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.705262    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"]
Sep 11 11:49:46.707677 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.707655    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-5a5ab7-6fc56dbdd7-csjs9"]
Sep 11 11:49:46.849529 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.849445    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:46.903821 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.903796    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"]
Sep 11 11:49:46.906050 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:49:46.906016    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b84e57a_c0b0_4490_94fe_a02c6f746031.slice/crio-93fa94b1a1c6658a72f2999d5799787c2316f408ec95abbfe337ca47204d6342 WatchSource:0}: Error finding container 93fa94b1a1c6658a72f2999d5799787c2316f408ec95abbfe337ca47204d6342: Status 404 returned error can't find the container with id 93fa94b1a1c6658a72f2999d5799787c2316f408ec95abbfe337ca47204d6342
Sep 11 11:49:46.907841 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.907820    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 11:49:46.952012 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:46.951963    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:47.697167 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:47.697074    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s" event={"ID":"9b84e57a-c0b0-4490-94fe-a02c6f746031","Type":"ContainerStarted","Data":"c3ddd6437a3de19d6400d019b6a90cee311b9becb096048a77e32f06a68b6451"}
Sep 11 11:49:47.697167 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:47.697135    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s" event={"ID":"9b84e57a-c0b0-4490-94fe-a02c6f746031","Type":"ContainerStarted","Data":"93fa94b1a1c6658a72f2999d5799787c2316f408ec95abbfe337ca47204d6342"}
Sep 11 11:49:47.697343 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:47.697279    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"
Sep 11 11:49:47.723877 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:47.723834    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s" podStartSLOduration=1.295305015 podStartE2EDuration="1.723818878s" totalImagesPullingTime="428.513863ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:49:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:49:47.72256689 +0000 UTC m=+928.143172918" watchObservedRunningTime="2026-09-11 11:49:47.723818878 +0000 UTC m=+928.144424907"
Sep 11 11:49:48.082139 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:48.082099    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa73d8ab-d829-47d9-84d2-f6fd20c954af" path="/var/lib/kubelet/pods/fa73d8ab-d829-47d9-84d2-f6fd20c954af/volumes"
Sep 11 11:49:48.200745 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:48.200710    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:51.341483 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:51.341451    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"]
Sep 11 11:49:51.410845 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:51.410806    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:49:53.708812 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:53.708784    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"
Sep 11 11:49:55.040752 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:55.040715    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:49:55.040752 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:55.040766    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:49:55.053444 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:55.053416    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:49:55.751972 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:49:55.751946    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:50:06.361556 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.361501    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq" podUID="a4308f1c-09b4-4658-949c-9858d9791554" containerName="main" containerID="cri-o://d459854cc75ef9938627703303fa575cda966c9183ee002361765d465e67f781" gracePeriod=45
Sep 11 11:50:06.613609 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.613552    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:50:06.699344 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.699311    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-model-cache\") pod \"a4308f1c-09b4-4658-949c-9858d9791554\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") "
Sep 11 11:50:06.699493 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.699406    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4308f1c-09b4-4658-949c-9858d9791554-tls-certs\") pod \"a4308f1c-09b4-4658-949c-9858d9791554\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") "
Sep 11 11:50:06.699536 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.699496    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-dshm\") pod \"a4308f1c-09b4-4658-949c-9858d9791554\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") "
Sep 11 11:50:06.699575 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.699542    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-tmp-dir\") pod \"a4308f1c-09b4-4658-949c-9858d9791554\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") "
Sep 11 11:50:06.699621 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.699603    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-model-cache" (OuterVolumeSpecName: "model-cache") pod "a4308f1c-09b4-4658-949c-9858d9791554" (UID: "a4308f1c-09b4-4658-949c-9858d9791554"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:50:06.699685 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.699607    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbgnl\" (UniqueName: \"kubernetes.io/projected/a4308f1c-09b4-4658-949c-9858d9791554-kube-api-access-fbgnl\") pod \"a4308f1c-09b4-4658-949c-9858d9791554\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") "
Sep 11 11:50:06.699743 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.699698    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-home\") pod \"a4308f1c-09b4-4658-949c-9858d9791554\" (UID: \"a4308f1c-09b4-4658-949c-9858d9791554\") "
Sep 11 11:50:06.699892 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.699864    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a4308f1c-09b4-4658-949c-9858d9791554" (UID: "a4308f1c-09b4-4658-949c-9858d9791554"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:50:06.699979 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.699946    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-home" (OuterVolumeSpecName: "home") pod "a4308f1c-09b4-4658-949c-9858d9791554" (UID: "a4308f1c-09b4-4658-949c-9858d9791554"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:50:06.700158 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.700136    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-tmp-dir\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:50:06.700158 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.700160    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-home\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:50:06.700304 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.700174    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-model-cache\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:50:06.701917 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.701896    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4308f1c-09b4-4658-949c-9858d9791554-kube-api-access-fbgnl" (OuterVolumeSpecName: "kube-api-access-fbgnl") pod "a4308f1c-09b4-4658-949c-9858d9791554" (UID: "a4308f1c-09b4-4658-949c-9858d9791554"). InnerVolumeSpecName "kube-api-access-fbgnl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:50:06.702116 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.702094    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-dshm" (OuterVolumeSpecName: "dshm") pod "a4308f1c-09b4-4658-949c-9858d9791554" (UID: "a4308f1c-09b4-4658-949c-9858d9791554"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:50:06.702192 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.702165    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4308f1c-09b4-4658-949c-9858d9791554-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a4308f1c-09b4-4658-949c-9858d9791554" (UID: "a4308f1c-09b4-4658-949c-9858d9791554"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:50:06.795976 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.795940    2575 generic.go:356] "Generic (PLEG): container finished" podID="a4308f1c-09b4-4658-949c-9858d9791554" containerID="d459854cc75ef9938627703303fa575cda966c9183ee002361765d465e67f781" exitCode=0
Sep 11 11:50:06.796124 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.795988    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"
Sep 11 11:50:06.796124 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.796022    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq" event={"ID":"a4308f1c-09b4-4658-949c-9858d9791554","Type":"ContainerDied","Data":"d459854cc75ef9938627703303fa575cda966c9183ee002361765d465e67f781"}
Sep 11 11:50:06.796124 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.796066    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq" event={"ID":"a4308f1c-09b4-4658-949c-9858d9791554","Type":"ContainerDied","Data":"c4770768ab567237fc6cfcfca819b9b097d4c7b73db8b25f0c2082136dce9ad6"}
Sep 11 11:50:06.796124 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.796083    2575 scope.go:117] "RemoveContainer" containerID="d459854cc75ef9938627703303fa575cda966c9183ee002361765d465e67f781"
Sep 11 11:50:06.800744 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.800725    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4308f1c-09b4-4658-949c-9858d9791554-tls-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:50:06.800744 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.800746    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4308f1c-09b4-4658-949c-9858d9791554-dshm\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:50:06.800884 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.800759    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbgnl\" (UniqueName: \"kubernetes.io/projected/a4308f1c-09b4-4658-949c-9858d9791554-kube-api-access-fbgnl\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:50:06.805479 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.805463    2575 scope.go:117] "RemoveContainer" containerID="d459854cc75ef9938627703303fa575cda966c9183ee002361765d465e67f781"
Sep 11 11:50:06.805783 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:50:06.805763    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d459854cc75ef9938627703303fa575cda966c9183ee002361765d465e67f781\": container with ID starting with d459854cc75ef9938627703303fa575cda966c9183ee002361765d465e67f781 not found: ID does not exist" containerID="d459854cc75ef9938627703303fa575cda966c9183ee002361765d465e67f781"
Sep 11 11:50:06.805840 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.805791    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d459854cc75ef9938627703303fa575cda966c9183ee002361765d465e67f781"} err="failed to get container status \"d459854cc75ef9938627703303fa575cda966c9183ee002361765d465e67f781\": rpc error: code = NotFound desc = could not find container \"d459854cc75ef9938627703303fa575cda966c9183ee002361765d465e67f781\": container with ID starting with d459854cc75ef9938627703303fa575cda966c9183ee002361765d465e67f781 not found: ID does not exist"
Sep 11 11:50:06.817454 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.817431    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"]
Sep 11 11:50:06.828237 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:06.824995    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-5a5ab7/autores-model-5a5ab7-kserve-d7f5f5b8c-rbzzq"]
Sep 11 11:50:07.426394 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:07.426363    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:07.533824 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:07.533794    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:08.082182 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:08.082148    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4308f1c-09b4-4658-949c-9858d9791554" path="/var/lib/kubelet/pods/a4308f1c-09b4-4658-949c-9858d9791554/volumes"
Sep 11 11:50:10.828506 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:10.828473    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:12.264618 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:12.264585    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"]
Sep 11 11:50:12.294438 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:12.294397    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:14.073339 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:14.073305    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:14.341337 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:14.341260    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:14.789252 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:14.789216    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:15.110248 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:15.110159    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:15.336712 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:15.336682    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:16.858757 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:16.858728    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"]
Sep 11 11:50:16.859222 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:16.858947    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s" podUID="9b84e57a-c0b0-4490-94fe-a02c6f746031" containerName="maas-api" containerID="cri-o://c3ddd6437a3de19d6400d019b6a90cee311b9becb096048a77e32f06a68b6451" gracePeriod=30
Sep 11 11:50:16.997982 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:16.997900    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:17.090547 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.090517    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:17.122093 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.122032    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"
Sep 11 11:50:17.193027 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.193002    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:17.304443 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.304408    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7sl75\" (UniqueName: \"kubernetes.io/projected/9b84e57a-c0b0-4490-94fe-a02c6f746031-kube-api-access-7sl75\") pod \"9b84e57a-c0b0-4490-94fe-a02c6f746031\" (UID: \"9b84e57a-c0b0-4490-94fe-a02c6f746031\") "
Sep 11 11:50:17.304611 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.304504    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b84e57a-c0b0-4490-94fe-a02c6f746031-openshift-service-ca\") pod \"9b84e57a-c0b0-4490-94fe-a02c6f746031\" (UID: \"9b84e57a-c0b0-4490-94fe-a02c6f746031\") "
Sep 11 11:50:17.304611 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.304565    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b84e57a-c0b0-4490-94fe-a02c6f746031-metrics-certs\") pod \"9b84e57a-c0b0-4490-94fe-a02c6f746031\" (UID: \"9b84e57a-c0b0-4490-94fe-a02c6f746031\") "
Sep 11 11:50:17.304611 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.304590    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b84e57a-c0b0-4490-94fe-a02c6f746031-maas-api-tls\") pod \"9b84e57a-c0b0-4490-94fe-a02c6f746031\" (UID: \"9b84e57a-c0b0-4490-94fe-a02c6f746031\") "
Sep 11 11:50:17.304908 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.304883    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b84e57a-c0b0-4490-94fe-a02c6f746031-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9b84e57a-c0b0-4490-94fe-a02c6f746031" (UID: "9b84e57a-c0b0-4490-94fe-a02c6f746031"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:50:17.306734 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.306701    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b84e57a-c0b0-4490-94fe-a02c6f746031-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9b84e57a-c0b0-4490-94fe-a02c6f746031" (UID: "9b84e57a-c0b0-4490-94fe-a02c6f746031"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:50:17.306834 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.306776    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b84e57a-c0b0-4490-94fe-a02c6f746031-kube-api-access-7sl75" (OuterVolumeSpecName: "kube-api-access-7sl75") pod "9b84e57a-c0b0-4490-94fe-a02c6f746031" (UID: "9b84e57a-c0b0-4490-94fe-a02c6f746031"). InnerVolumeSpecName "kube-api-access-7sl75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:50:17.306834 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.306777    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b84e57a-c0b0-4490-94fe-a02c6f746031-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9b84e57a-c0b0-4490-94fe-a02c6f746031" (UID: "9b84e57a-c0b0-4490-94fe-a02c6f746031"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:50:17.329112 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.329082    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:17.405620 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.405550    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7sl75\" (UniqueName: \"kubernetes.io/projected/9b84e57a-c0b0-4490-94fe-a02c6f746031-kube-api-access-7sl75\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:50:17.405620 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.405574    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b84e57a-c0b0-4490-94fe-a02c6f746031-openshift-service-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:50:17.405620 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.405583    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b84e57a-c0b0-4490-94fe-a02c6f746031-metrics-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:50:17.405620 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.405592    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b84e57a-c0b0-4490-94fe-a02c6f746031-maas-api-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:50:17.855595 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.855560    2575 generic.go:356] "Generic (PLEG): container finished" podID="9b84e57a-c0b0-4490-94fe-a02c6f746031" containerID="c3ddd6437a3de19d6400d019b6a90cee311b9becb096048a77e32f06a68b6451" exitCode=0
Sep 11 11:50:17.855784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.855602    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"
Sep 11 11:50:17.855784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.855659    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s" event={"ID":"9b84e57a-c0b0-4490-94fe-a02c6f746031","Type":"ContainerDied","Data":"c3ddd6437a3de19d6400d019b6a90cee311b9becb096048a77e32f06a68b6451"}
Sep 11 11:50:17.855784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.855711    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s" event={"ID":"9b84e57a-c0b0-4490-94fe-a02c6f746031","Type":"ContainerDied","Data":"93fa94b1a1c6658a72f2999d5799787c2316f408ec95abbfe337ca47204d6342"}
Sep 11 11:50:17.855784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.855741    2575 scope.go:117] "RemoveContainer" containerID="c3ddd6437a3de19d6400d019b6a90cee311b9becb096048a77e32f06a68b6451"
Sep 11 11:50:17.867566 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.867532    2575 scope.go:117] "RemoveContainer" containerID="c3ddd6437a3de19d6400d019b6a90cee311b9becb096048a77e32f06a68b6451"
Sep 11 11:50:17.868173 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:50:17.868141    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3ddd6437a3de19d6400d019b6a90cee311b9becb096048a77e32f06a68b6451\": container with ID starting with c3ddd6437a3de19d6400d019b6a90cee311b9becb096048a77e32f06a68b6451 not found: ID does not exist" containerID="c3ddd6437a3de19d6400d019b6a90cee311b9becb096048a77e32f06a68b6451"
Sep 11 11:50:17.868274 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.868182    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3ddd6437a3de19d6400d019b6a90cee311b9becb096048a77e32f06a68b6451"} err="failed to get container status \"c3ddd6437a3de19d6400d019b6a90cee311b9becb096048a77e32f06a68b6451\": rpc error: code = NotFound desc = could not find container \"c3ddd6437a3de19d6400d019b6a90cee311b9becb096048a77e32f06a68b6451\": container with ID starting with c3ddd6437a3de19d6400d019b6a90cee311b9becb096048a77e32f06a68b6451 not found: ID does not exist"
Sep 11 11:50:17.898432 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.898402    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"]
Sep 11 11:50:17.903347 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.903324    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d507e5c9-75b989bd58-6hj2s"]
Sep 11 11:50:17.933941 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:17.933911    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:18.060603 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:18.060572    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:18.081816 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:18.081791    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b84e57a-c0b0-4490-94fe-a02c6f746031" path="/var/lib/kubelet/pods/9b84e57a-c0b0-4490-94fe-a02c6f746031/volumes"
Sep 11 11:50:18.200979 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:18.200891    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:19.153582 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.153555    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"]
Sep 11 11:50:19.154234 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.154168    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b84e57a-c0b0-4490-94fe-a02c6f746031" containerName="maas-api"
Sep 11 11:50:19.154234 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.154194    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b84e57a-c0b0-4490-94fe-a02c6f746031" containerName="maas-api"
Sep 11 11:50:19.154234 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.154227    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4308f1c-09b4-4658-949c-9858d9791554" containerName="main"
Sep 11 11:50:19.154234 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.154233    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4308f1c-09b4-4658-949c-9858d9791554" containerName="main"
Sep 11 11:50:19.154686 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.154326    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4308f1c-09b4-4658-949c-9858d9791554" containerName="main"
Sep 11 11:50:19.154686 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.154335    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b84e57a-c0b0-4490-94fe-a02c6f746031" containerName="maas-api"
Sep 11 11:50:19.158933 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.158909    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"
Sep 11 11:50:19.161368 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.161348    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-c92551-serving-cert\""
Sep 11 11:50:19.171383 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.171361    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"]
Sep 11 11:50:19.191343 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.191312    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:19.224531 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.224487    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpxsw\" (UniqueName: \"kubernetes.io/projected/5cab9392-2eb4-4d58-99d7-1022403466c1-kube-api-access-mpxsw\") pod \"maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng\" (UID: \"5cab9392-2eb4-4d58-99d7-1022403466c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"
Sep 11 11:50:19.224731 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.224588    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cab9392-2eb4-4d58-99d7-1022403466c1-openshift-service-ca\") pod \"maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng\" (UID: \"5cab9392-2eb4-4d58-99d7-1022403466c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"
Sep 11 11:50:19.224731 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.224703    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5cab9392-2eb4-4d58-99d7-1022403466c1-maas-api-tls\") pod \"maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng\" (UID: \"5cab9392-2eb4-4d58-99d7-1022403466c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"
Sep 11 11:50:19.224856 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.224773    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5cab9392-2eb4-4d58-99d7-1022403466c1-metrics-certs\") pod \"maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng\" (UID: \"5cab9392-2eb4-4d58-99d7-1022403466c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"
Sep 11 11:50:19.282995 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.282956    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:19.325836 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.325805    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpxsw\" (UniqueName: \"kubernetes.io/projected/5cab9392-2eb4-4d58-99d7-1022403466c1-kube-api-access-mpxsw\") pod \"maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng\" (UID: \"5cab9392-2eb4-4d58-99d7-1022403466c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"
Sep 11 11:50:19.325977 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.325855    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cab9392-2eb4-4d58-99d7-1022403466c1-openshift-service-ca\") pod \"maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng\" (UID: \"5cab9392-2eb4-4d58-99d7-1022403466c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"
Sep 11 11:50:19.325977 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.325899    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5cab9392-2eb4-4d58-99d7-1022403466c1-maas-api-tls\") pod \"maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng\" (UID: \"5cab9392-2eb4-4d58-99d7-1022403466c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"
Sep 11 11:50:19.325977 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.325936    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5cab9392-2eb4-4d58-99d7-1022403466c1-metrics-certs\") pod \"maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng\" (UID: \"5cab9392-2eb4-4d58-99d7-1022403466c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"
Sep 11 11:50:19.326083 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:50:19.326049    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-serving-cert: secret "maas-api-e2e-explicit-c92551-serving-cert" not found
Sep 11 11:50:19.326137 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:50:19.326126    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cab9392-2eb4-4d58-99d7-1022403466c1-maas-api-tls podName:5cab9392-2eb4-4d58-99d7-1022403466c1 nodeName:}" failed. No retries permitted until 2026-09-11 11:50:19.82610359 +0000 UTC m=+960.246709595 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5cab9392-2eb4-4d58-99d7-1022403466c1-maas-api-tls") pod "maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng" (UID: "5cab9392-2eb4-4d58-99d7-1022403466c1") : secret "maas-api-e2e-explicit-c92551-serving-cert" not found
Sep 11 11:50:19.326547 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.326529    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cab9392-2eb4-4d58-99d7-1022403466c1-openshift-service-ca\") pod \"maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng\" (UID: \"5cab9392-2eb4-4d58-99d7-1022403466c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"
Sep 11 11:50:19.328577 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.328554    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5cab9392-2eb4-4d58-99d7-1022403466c1-metrics-certs\") pod \"maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng\" (UID: \"5cab9392-2eb4-4d58-99d7-1022403466c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"
Sep 11 11:50:19.337082 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.337061    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpxsw\" (UniqueName: \"kubernetes.io/projected/5cab9392-2eb4-4d58-99d7-1022403466c1-kube-api-access-mpxsw\") pod \"maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng\" (UID: \"5cab9392-2eb4-4d58-99d7-1022403466c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"
Sep 11 11:50:19.422678 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.422562    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:19.561741 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.561694    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:19.818193 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.818152    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:19.831469 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.831433    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5cab9392-2eb4-4d58-99d7-1022403466c1-maas-api-tls\") pod \"maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng\" (UID: \"5cab9392-2eb4-4d58-99d7-1022403466c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"
Sep 11 11:50:19.833992 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.833972    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5cab9392-2eb4-4d58-99d7-1022403466c1-maas-api-tls\") pod \"maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng\" (UID: \"5cab9392-2eb4-4d58-99d7-1022403466c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"
Sep 11 11:50:19.918406 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:19.918368    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:20.070085 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:20.070001    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"
Sep 11 11:50:20.347869 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:20.347845    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"]
Sep 11 11:50:20.350106 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:50:20.350081    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cab9392_2eb4_4d58_99d7_1022403466c1.slice/crio-7378a81febdbc838a861d4923e1858e788a0900826bb951e0b7b6d099b1d3733 WatchSource:0}: Error finding container 7378a81febdbc838a861d4923e1858e788a0900826bb951e0b7b6d099b1d3733: Status 404 returned error can't find the container with id 7378a81febdbc838a861d4923e1858e788a0900826bb951e0b7b6d099b1d3733
Sep 11 11:50:20.872993 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:20.872967    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng" event={"ID":"5cab9392-2eb4-4d58-99d7-1022403466c1","Type":"ContainerStarted","Data":"7378a81febdbc838a861d4923e1858e788a0900826bb951e0b7b6d099b1d3733"}
Sep 11 11:50:21.879940 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:21.879905    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng" event={"ID":"5cab9392-2eb4-4d58-99d7-1022403466c1","Type":"ContainerStarted","Data":"234af6a180bdf8956f91bc3cb444ba26649f73272bd12f6e754529a0092745a2"}
Sep 11 11:50:21.880347 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:21.880056    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"
Sep 11 11:50:21.898483 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:21.898436    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng" podStartSLOduration=2.381195512 podStartE2EDuration="2.898421402s" totalImagesPullingTime="517.22589ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:50:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:50:21.897586842 +0000 UTC m=+962.318192870" watchObservedRunningTime="2026-09-11 11:50:21.898421402 +0000 UTC m=+962.319027429"
Sep 11 11:50:25.330411 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:25.330376    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:25.434902 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:25.434868    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:25.530968 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:25.530937    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:25.631120 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:25.631027    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:26.438719 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:26.438688    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:27.288664 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.288610    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s" podUID="d673fd55-264f-4c76-8c78-c2af4a4ca4ad" containerName="main" containerID="cri-o://909bb000dcbba4f1a6c19a5f2d5bc58acc1a6b8cb29038a4afcd196c1663085e" gracePeriod=45
Sep 11 11:50:27.536799 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.536775    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:50:27.607298 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.607219    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-tmp-dir\") pod \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") "
Sep 11 11:50:27.607298 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.607284    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-dshm\") pod \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") "
Sep 11 11:50:27.607496 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.607320    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mt6d\" (UniqueName: \"kubernetes.io/projected/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-kube-api-access-4mt6d\") pod \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") "
Sep 11 11:50:27.607496 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.607354    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-home\") pod \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") "
Sep 11 11:50:27.607496 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.607414    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-model-cache\") pod \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") "
Sep 11 11:50:27.607496 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.607452    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d673fd55-264f-4c76-8c78-c2af4a4ca4ad" (UID: "d673fd55-264f-4c76-8c78-c2af4a4ca4ad"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:50:27.607496 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.607491    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-tls-certs\") pod \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\" (UID: \"d673fd55-264f-4c76-8c78-c2af4a4ca4ad\") "
Sep 11 11:50:27.607817 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.607684    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-model-cache" (OuterVolumeSpecName: "model-cache") pod "d673fd55-264f-4c76-8c78-c2af4a4ca4ad" (UID: "d673fd55-264f-4c76-8c78-c2af4a4ca4ad"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:50:27.607817 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.607714    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-home" (OuterVolumeSpecName: "home") pod "d673fd55-264f-4c76-8c78-c2af4a4ca4ad" (UID: "d673fd55-264f-4c76-8c78-c2af4a4ca4ad"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:50:27.607920 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.607883    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-home\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:50:27.607920 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.607903    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-model-cache\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:50:27.608005 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.607940    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-tmp-dir\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:50:27.609727 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.609698    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-kube-api-access-4mt6d" (OuterVolumeSpecName: "kube-api-access-4mt6d") pod "d673fd55-264f-4c76-8c78-c2af4a4ca4ad" (UID: "d673fd55-264f-4c76-8c78-c2af4a4ca4ad"). InnerVolumeSpecName "kube-api-access-4mt6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:50:27.609826 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.609724    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d673fd55-264f-4c76-8c78-c2af4a4ca4ad" (UID: "d673fd55-264f-4c76-8c78-c2af4a4ca4ad"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:50:27.609826 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.609748    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-dshm" (OuterVolumeSpecName: "dshm") pod "d673fd55-264f-4c76-8c78-c2af4a4ca4ad" (UID: "d673fd55-264f-4c76-8c78-c2af4a4ca4ad"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:50:27.709223 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.709195    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-tls-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:50:27.709223 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.709218    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-dshm\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:50:27.709223 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.709227    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mt6d\" (UniqueName: \"kubernetes.io/projected/d673fd55-264f-4c76-8c78-c2af4a4ca4ad-kube-api-access-4mt6d\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:50:27.891233 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.891160    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"
Sep 11 11:50:27.912652 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.912602    2575 generic.go:356] "Generic (PLEG): container finished" podID="d673fd55-264f-4c76-8c78-c2af4a4ca4ad" containerID="909bb000dcbba4f1a6c19a5f2d5bc58acc1a6b8cb29038a4afcd196c1663085e" exitCode=0
Sep 11 11:50:27.912787 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.912665    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"
Sep 11 11:50:27.912787 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.912668    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s" event={"ID":"d673fd55-264f-4c76-8c78-c2af4a4ca4ad","Type":"ContainerDied","Data":"909bb000dcbba4f1a6c19a5f2d5bc58acc1a6b8cb29038a4afcd196c1663085e"}
Sep 11 11:50:27.912932 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.912777    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s" event={"ID":"d673fd55-264f-4c76-8c78-c2af4a4ca4ad","Type":"ContainerDied","Data":"f710b038869a25160bf7eddfbc99dcfd56737582cb8eb37e4d8324ca46c0a310"}
Sep 11 11:50:27.912932 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.912811    2575 scope.go:117] "RemoveContainer" containerID="909bb000dcbba4f1a6c19a5f2d5bc58acc1a6b8cb29038a4afcd196c1663085e"
Sep 11 11:50:27.922563 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.922542    2575 scope.go:117] "RemoveContainer" containerID="909bb000dcbba4f1a6c19a5f2d5bc58acc1a6b8cb29038a4afcd196c1663085e"
Sep 11 11:50:27.922878 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:50:27.922857    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"909bb000dcbba4f1a6c19a5f2d5bc58acc1a6b8cb29038a4afcd196c1663085e\": container with ID starting with 909bb000dcbba4f1a6c19a5f2d5bc58acc1a6b8cb29038a4afcd196c1663085e not found: ID does not exist" containerID="909bb000dcbba4f1a6c19a5f2d5bc58acc1a6b8cb29038a4afcd196c1663085e"
Sep 11 11:50:27.922963 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.922891    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"909bb000dcbba4f1a6c19a5f2d5bc58acc1a6b8cb29038a4afcd196c1663085e"} err="failed to get container status \"909bb000dcbba4f1a6c19a5f2d5bc58acc1a6b8cb29038a4afcd196c1663085e\": rpc error: code = NotFound desc = could not find container \"909bb000dcbba4f1a6c19a5f2d5bc58acc1a6b8cb29038a4afcd196c1663085e\": container with ID starting with 909bb000dcbba4f1a6c19a5f2d5bc58acc1a6b8cb29038a4afcd196c1663085e not found: ID does not exist"
Sep 11 11:50:27.938178 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.938147    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"]
Sep 11 11:50:27.942589 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:27.942562    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-31a432b7-kserve-7f596ddf7d-pwc7s"]
Sep 11 11:50:28.081618 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:28.081585    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d673fd55-264f-4c76-8c78-c2af4a4ca4ad" path="/var/lib/kubelet/pods/d673fd55-264f-4c76-8c78-c2af4a4ca4ad/volumes"
Sep 11 11:50:42.916022 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:42.915937    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:44.873593 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:44.873561    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:52.005889 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:52.005856    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:57.920321 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:57.920282    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:59.129198 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.129162    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:59.233474 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.233440    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"]
Sep 11 11:50:59.233901 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.233886    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d673fd55-264f-4c76-8c78-c2af4a4ca4ad" containerName="main"
Sep 11 11:50:59.233901 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.233902    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d673fd55-264f-4c76-8c78-c2af4a4ca4ad" containerName="main"
Sep 11 11:50:59.234012 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.233983    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d673fd55-264f-4c76-8c78-c2af4a4ca4ad" containerName="main"
Sep 11 11:50:59.235989 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.235962    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:59.237183 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.237165    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:50:59.240622 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.240600    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-c92551\"/\"kube-root-ca.crt\""
Sep 11 11:50:59.240622 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.240613    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-c92551\"/\"openshift-service-ca.crt\""
Sep 11 11:50:59.240622 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.240616    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-c92551\"/\"explicit-model-c92551-kserve-self-signed-certs\""
Sep 11 11:50:59.240848 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.240669    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-c92551\"/\"default-dockercfg-p2v99\""
Sep 11 11:50:59.252771 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.252751    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"]
Sep 11 11:50:59.321169 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.321140    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-model-cache\") pod \"explicit-model-c92551-kserve-df4bc6c9-6hn8s\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") " pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:50:59.321339 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.321197    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-home\") pod \"explicit-model-c92551-kserve-df4bc6c9-6hn8s\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") " pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:50:59.321339 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.321219    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-dshm\") pod \"explicit-model-c92551-kserve-df4bc6c9-6hn8s\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") " pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:50:59.321339 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.321239    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-tls-certs\") pod \"explicit-model-c92551-kserve-df4bc6c9-6hn8s\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") " pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:50:59.321339 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.321267    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-tmp-dir\") pod \"explicit-model-c92551-kserve-df4bc6c9-6hn8s\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") " pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:50:59.321339 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.321335    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xtr6\" (UniqueName: \"kubernetes.io/projected/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-kube-api-access-8xtr6\") pod \"explicit-model-c92551-kserve-df4bc6c9-6hn8s\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") " pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:50:59.325404 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.325371    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:50:59.422718 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.422615    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-tmp-dir\") pod \"explicit-model-c92551-kserve-df4bc6c9-6hn8s\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") " pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:50:59.422876 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.422740    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xtr6\" (UniqueName: \"kubernetes.io/projected/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-kube-api-access-8xtr6\") pod \"explicit-model-c92551-kserve-df4bc6c9-6hn8s\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") " pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:50:59.422876 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.422798    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-model-cache\") pod \"explicit-model-c92551-kserve-df4bc6c9-6hn8s\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") " pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:50:59.422876 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.422850    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-home\") pod \"explicit-model-c92551-kserve-df4bc6c9-6hn8s\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") " pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:50:59.423057 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.422876    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-dshm\") pod \"explicit-model-c92551-kserve-df4bc6c9-6hn8s\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") " pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:50:59.423057 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.422904    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-tls-certs\") pod \"explicit-model-c92551-kserve-df4bc6c9-6hn8s\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") " pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:50:59.423057 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.423044    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-tmp-dir\") pod \"explicit-model-c92551-kserve-df4bc6c9-6hn8s\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") " pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:50:59.423190 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.423158    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-model-cache\") pod \"explicit-model-c92551-kserve-df4bc6c9-6hn8s\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") " pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:50:59.423229 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.423197    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-home\") pod \"explicit-model-c92551-kserve-df4bc6c9-6hn8s\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") " pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:50:59.425276 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.425243    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-dshm\") pod \"explicit-model-c92551-kserve-df4bc6c9-6hn8s\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") " pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:50:59.425529 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.425514    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-tls-certs\") pod \"explicit-model-c92551-kserve-df4bc6c9-6hn8s\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") " pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:50:59.431465 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.431437    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xtr6\" (UniqueName: \"kubernetes.io/projected/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-kube-api-access-8xtr6\") pod \"explicit-model-c92551-kserve-df4bc6c9-6hn8s\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") " pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:50:59.546231 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.546190    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:50:59.821620 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:50:59.821598    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"]
Sep 11 11:50:59.823768 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:50:59.823742    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d315cc9_bbcc_4b1e_9ba3_c6aefd406bad.slice/crio-0ea0f029043837c8b81cf33d3c39106a4fd436c431ed7fee0f1f2c9d992f0677 WatchSource:0}: Error finding container 0ea0f029043837c8b81cf33d3c39106a4fd436c431ed7fee0f1f2c9d992f0677: Status 404 returned error can't find the container with id 0ea0f029043837c8b81cf33d3c39106a4fd436c431ed7fee0f1f2c9d992f0677
Sep 11 11:51:00.074952 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:00.074858    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s" event={"ID":"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad","Type":"ContainerStarted","Data":"9f606abd441ad38fc284ce380df5acd2f8a502b752199bcff9a29fc7698d6d2f"}
Sep 11 11:51:00.074952 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:00.074910    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s" event={"ID":"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad","Type":"ContainerStarted","Data":"0ea0f029043837c8b81cf33d3c39106a4fd436c431ed7fee0f1f2c9d992f0677"}
Sep 11 11:51:00.103206 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:00.103158    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s" podStartSLOduration=1.103142762 podStartE2EDuration="1.103142762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:50:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:51:00.101077832 +0000 UTC m=+1000.521683886" watchObservedRunningTime="2026-09-11 11:51:00.103142762 +0000 UTC m=+1000.523748788"
Sep 11 11:51:09.546826 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:09.546794    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:51:09.547270 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:09.546859    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:51:09.559381 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:09.559356    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:51:10.134826 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:10.134800    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:51:11.864245 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:11.864214    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:13.818387 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:13.818349    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"]
Sep 11 11:51:13.842571 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:13.842543    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:14.022896 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:14.022859    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:20.079521 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:20.079478    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:20.176089 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:20.176047    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:20.273277 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:20.273241    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:22.636968 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:22.636937    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:22.718245 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:22.718213    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:24.449248 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:24.449214    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"]
Sep 11 11:51:24.449758 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:24.449551    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng" podUID="5cab9392-2eb4-4d58-99d7-1022403466c1" containerName="maas-api" containerID="cri-o://234af6a180bdf8956f91bc3cb444ba26649f73272bd12f6e754529a0092745a2" gracePeriod=30
Sep 11 11:51:24.480413 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:24.480380    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:24.582166 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:24.582133    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:24.686113 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:24.686076    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:24.701252 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:24.701202    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"
Sep 11 11:51:24.770806 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:24.770776    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5cab9392-2eb4-4d58-99d7-1022403466c1-metrics-certs\") pod \"5cab9392-2eb4-4d58-99d7-1022403466c1\" (UID: \"5cab9392-2eb4-4d58-99d7-1022403466c1\") "
Sep 11 11:51:24.770961 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:24.770824    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5cab9392-2eb4-4d58-99d7-1022403466c1-maas-api-tls\") pod \"5cab9392-2eb4-4d58-99d7-1022403466c1\" (UID: \"5cab9392-2eb4-4d58-99d7-1022403466c1\") "
Sep 11 11:51:24.771004 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:24.770957    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cab9392-2eb4-4d58-99d7-1022403466c1-openshift-service-ca\") pod \"5cab9392-2eb4-4d58-99d7-1022403466c1\" (UID: \"5cab9392-2eb4-4d58-99d7-1022403466c1\") "
Sep 11 11:51:24.771004 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:24.770985    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpxsw\" (UniqueName: \"kubernetes.io/projected/5cab9392-2eb4-4d58-99d7-1022403466c1-kube-api-access-mpxsw\") pod \"5cab9392-2eb4-4d58-99d7-1022403466c1\" (UID: \"5cab9392-2eb4-4d58-99d7-1022403466c1\") "
Sep 11 11:51:24.771299 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:24.771266    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cab9392-2eb4-4d58-99d7-1022403466c1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5cab9392-2eb4-4d58-99d7-1022403466c1" (UID: "5cab9392-2eb4-4d58-99d7-1022403466c1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:51:24.773088 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:24.773062    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5cab9392-2eb4-4d58-99d7-1022403466c1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5cab9392-2eb4-4d58-99d7-1022403466c1" (UID: "5cab9392-2eb4-4d58-99d7-1022403466c1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:51:24.773211 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:24.773124    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5cab9392-2eb4-4d58-99d7-1022403466c1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5cab9392-2eb4-4d58-99d7-1022403466c1" (UID: "5cab9392-2eb4-4d58-99d7-1022403466c1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:51:24.773211 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:24.773182    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cab9392-2eb4-4d58-99d7-1022403466c1-kube-api-access-mpxsw" (OuterVolumeSpecName: "kube-api-access-mpxsw") pod "5cab9392-2eb4-4d58-99d7-1022403466c1" (UID: "5cab9392-2eb4-4d58-99d7-1022403466c1"). InnerVolumeSpecName "kube-api-access-mpxsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:51:24.784550 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:24.784516    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:24.871938 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:24.871900    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5cab9392-2eb4-4d58-99d7-1022403466c1-openshift-service-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:51:24.871938 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:24.871926    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpxsw\" (UniqueName: \"kubernetes.io/projected/5cab9392-2eb4-4d58-99d7-1022403466c1-kube-api-access-mpxsw\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:51:24.871938 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:24.871942    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5cab9392-2eb4-4d58-99d7-1022403466c1-metrics-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:51:24.872172 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:24.871955    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5cab9392-2eb4-4d58-99d7-1022403466c1-maas-api-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:51:25.099212 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:25.099176    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:25.204903 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:25.204872    2575 generic.go:356] "Generic (PLEG): container finished" podID="5cab9392-2eb4-4d58-99d7-1022403466c1" containerID="234af6a180bdf8956f91bc3cb444ba26649f73272bd12f6e754529a0092745a2" exitCode=0
Sep 11 11:51:25.205079 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:25.204914    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"
Sep 11 11:51:25.205079 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:25.204958    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng" event={"ID":"5cab9392-2eb4-4d58-99d7-1022403466c1","Type":"ContainerDied","Data":"234af6a180bdf8956f91bc3cb444ba26649f73272bd12f6e754529a0092745a2"}
Sep 11 11:51:25.205079 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:25.205007    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng" event={"ID":"5cab9392-2eb4-4d58-99d7-1022403466c1","Type":"ContainerDied","Data":"7378a81febdbc838a861d4923e1858e788a0900826bb951e0b7b6d099b1d3733"}
Sep 11 11:51:25.205079 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:25.205029    2575 scope.go:117] "RemoveContainer" containerID="234af6a180bdf8956f91bc3cb444ba26649f73272bd12f6e754529a0092745a2"
Sep 11 11:51:25.215272 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:25.215254    2575 scope.go:117] "RemoveContainer" containerID="234af6a180bdf8956f91bc3cb444ba26649f73272bd12f6e754529a0092745a2"
Sep 11 11:51:25.215517 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:51:25.215497    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"234af6a180bdf8956f91bc3cb444ba26649f73272bd12f6e754529a0092745a2\": container with ID starting with 234af6a180bdf8956f91bc3cb444ba26649f73272bd12f6e754529a0092745a2 not found: ID does not exist" containerID="234af6a180bdf8956f91bc3cb444ba26649f73272bd12f6e754529a0092745a2"
Sep 11 11:51:25.215581 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:25.215530    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"234af6a180bdf8956f91bc3cb444ba26649f73272bd12f6e754529a0092745a2"} err="failed to get container status \"234af6a180bdf8956f91bc3cb444ba26649f73272bd12f6e754529a0092745a2\": rpc error: code = NotFound desc = could not find container \"234af6a180bdf8956f91bc3cb444ba26649f73272bd12f6e754529a0092745a2\": container with ID starting with 234af6a180bdf8956f91bc3cb444ba26649f73272bd12f6e754529a0092745a2 not found: ID does not exist"
Sep 11 11:51:25.227105 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:25.227086    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"]
Sep 11 11:51:25.231671 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:25.231627    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c92551-69bf68bdf6-8bdng"]
Sep 11 11:51:26.081791 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:26.081757    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cab9392-2eb4-4d58-99d7-1022403466c1" path="/var/lib/kubelet/pods/5cab9392-2eb4-4d58-99d7-1022403466c1/volumes"
Sep 11 11:51:28.838390 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:28.838348    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s" podUID="1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad" containerName="main" containerID="cri-o://9f606abd441ad38fc284ce380df5acd2f8a502b752199bcff9a29fc7698d6d2f" gracePeriod=45
Sep 11 11:51:29.093984 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.093926    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:51:29.220115 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.220086    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-tls-certs\") pod \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") "
Sep 11 11:51:29.220115 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.220129    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-dshm\") pod \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") "
Sep 11 11:51:29.220362 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.220200    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-tmp-dir\") pod \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") "
Sep 11 11:51:29.220362 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.220250    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-model-cache\") pod \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") "
Sep 11 11:51:29.220362 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.220282    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-home\") pod \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") "
Sep 11 11:51:29.220362 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.220308    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xtr6\" (UniqueName: \"kubernetes.io/projected/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-kube-api-access-8xtr6\") pod \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\" (UID: \"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad\") "
Sep 11 11:51:29.220577 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.220531    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-home" (OuterVolumeSpecName: "home") pod "1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad" (UID: "1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:51:29.220577 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.220543    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad" (UID: "1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:51:29.220577 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.220559    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-model-cache" (OuterVolumeSpecName: "model-cache") pod "1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad" (UID: "1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:51:29.222529 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.222504    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-dshm" (OuterVolumeSpecName: "dshm") pod "1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad" (UID: "1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:51:29.222660 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.222576    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad" (UID: "1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:51:29.222718 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.222682    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-kube-api-access-8xtr6" (OuterVolumeSpecName: "kube-api-access-8xtr6") pod "1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad" (UID: "1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad"). InnerVolumeSpecName "kube-api-access-8xtr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:51:29.226868 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.226845    2575 generic.go:356] "Generic (PLEG): container finished" podID="1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad" containerID="9f606abd441ad38fc284ce380df5acd2f8a502b752199bcff9a29fc7698d6d2f" exitCode=0
Sep 11 11:51:29.226974 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.226941    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"
Sep 11 11:51:29.226974 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.226955    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s" event={"ID":"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad","Type":"ContainerDied","Data":"9f606abd441ad38fc284ce380df5acd2f8a502b752199bcff9a29fc7698d6d2f"}
Sep 11 11:51:29.227054 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.226994    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s" event={"ID":"1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad","Type":"ContainerDied","Data":"0ea0f029043837c8b81cf33d3c39106a4fd436c431ed7fee0f1f2c9d992f0677"}
Sep 11 11:51:29.227054 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.227014    2575 scope.go:117] "RemoveContainer" containerID="9f606abd441ad38fc284ce380df5acd2f8a502b752199bcff9a29fc7698d6d2f"
Sep 11 11:51:29.242675 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.242618    2575 scope.go:117] "RemoveContainer" containerID="9f606abd441ad38fc284ce380df5acd2f8a502b752199bcff9a29fc7698d6d2f"
Sep 11 11:51:29.242941 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:51:29.242923    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f606abd441ad38fc284ce380df5acd2f8a502b752199bcff9a29fc7698d6d2f\": container with ID starting with 9f606abd441ad38fc284ce380df5acd2f8a502b752199bcff9a29fc7698d6d2f not found: ID does not exist" containerID="9f606abd441ad38fc284ce380df5acd2f8a502b752199bcff9a29fc7698d6d2f"
Sep 11 11:51:29.242994 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.242949    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f606abd441ad38fc284ce380df5acd2f8a502b752199bcff9a29fc7698d6d2f"} err="failed to get container status \"9f606abd441ad38fc284ce380df5acd2f8a502b752199bcff9a29fc7698d6d2f\": rpc error: code = NotFound desc = could not find container \"9f606abd441ad38fc284ce380df5acd2f8a502b752199bcff9a29fc7698d6d2f\": container with ID starting with 9f606abd441ad38fc284ce380df5acd2f8a502b752199bcff9a29fc7698d6d2f not found: ID does not exist"
Sep 11 11:51:29.253408 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.253384    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"]
Sep 11 11:51:29.258938 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.258917    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-c92551/explicit-model-c92551-kserve-df4bc6c9-6hn8s"]
Sep 11 11:51:29.321719 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.321693    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-tmp-dir\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:51:29.321719 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.321715    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-model-cache\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:51:29.321719 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.321722    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-home\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:51:29.321908 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.321733    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xtr6\" (UniqueName: \"kubernetes.io/projected/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-kube-api-access-8xtr6\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:51:29.321908 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.321743    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-tls-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:51:29.321908 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:29.321753    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad-dshm\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:51:30.082542 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:30.082510    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad" path="/var/lib/kubelet/pods/1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad/volumes"
Sep 11 11:51:31.502209 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:31.502172    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:31.602262 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:31.602234    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:31.682621 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:31.682583    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:32.057450 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:32.057409    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:32.400056 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:32.399957    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:32.885373 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:32.885332    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:33.738800 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:33.738768    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:33.835894 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:33.835865    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:33.935646 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:33.935597    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:37.774010 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:37.773974    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:38.320903 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:38.320865    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:38.421337 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:38.421299    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:38.815504 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:38.815474    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:38.896309 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:38.896278    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:40.479950 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:40.479908    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:46.349668 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:46.349617    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:46.450187 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:46.450145    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:47.053856 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:47.053819    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:50.813930 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:50.813897    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"]
Sep 11 11:51:50.814527 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:50.814508    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad" containerName="main"
Sep 11 11:51:50.814617 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:50.814530    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad" containerName="main"
Sep 11 11:51:50.814617 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:50.814565    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cab9392-2eb4-4d58-99d7-1022403466c1" containerName="maas-api"
Sep 11 11:51:50.814617 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:50.814573    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cab9392-2eb4-4d58-99d7-1022403466c1" containerName="maas-api"
Sep 11 11:51:50.814810 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:50.814698    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cab9392-2eb4-4d58-99d7-1022403466c1" containerName="maas-api"
Sep 11 11:51:50.814810 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:50.814713    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d315cc9-bbcc-4b1e-9ba3-c6aefd406bad" containerName="main"
Sep 11 11:51:50.817948 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:50.817929    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:51:50.821556 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:50.821535    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-4503d2\"/\"openshift-service-ca.crt\""
Sep 11 11:51:50.821702 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:50.821588    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-4503d2\"/\"kube-root-ca.crt\""
Sep 11 11:51:50.821702 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:50.821535    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-4503d2\"/\"default-dockercfg-wfc7k\""
Sep 11 11:51:50.821702 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:50.821664    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-4503d2\"/\"noten-model-4503d2-kserve-self-signed-certs\""
Sep 11 11:51:50.825918 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:50.825896    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"]
Sep 11 11:51:50.840077 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:50.840045    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:50.919510 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:50.919475    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-model-cache\") pod \"noten-model-4503d2-kserve-7766d97bd7-7qffj\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") " pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:51:50.919510 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:50.919516    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcm9t\" (UniqueName: \"kubernetes.io/projected/daca7a4e-29b6-425d-b8b3-0f665bdb229c-kube-api-access-lcm9t\") pod \"noten-model-4503d2-kserve-7766d97bd7-7qffj\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") " pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:51:50.919826 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:50.919625    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-dshm\") pod \"noten-model-4503d2-kserve-7766d97bd7-7qffj\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") " pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:51:50.919826 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:50.919705    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-home\") pod \"noten-model-4503d2-kserve-7766d97bd7-7qffj\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") " pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:51:50.919826 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:50.919748    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-tmp-dir\") pod \"noten-model-4503d2-kserve-7766d97bd7-7qffj\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") " pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:51:50.919826 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:50.919775    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daca7a4e-29b6-425d-b8b3-0f665bdb229c-tls-certs\") pod \"noten-model-4503d2-kserve-7766d97bd7-7qffj\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") " pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:51:51.021079 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:51.021042    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-dshm\") pod \"noten-model-4503d2-kserve-7766d97bd7-7qffj\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") " pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:51:51.021079 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:51.021078    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-home\") pod \"noten-model-4503d2-kserve-7766d97bd7-7qffj\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") " pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:51:51.021318 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:51.021100    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-tmp-dir\") pod \"noten-model-4503d2-kserve-7766d97bd7-7qffj\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") " pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:51:51.021318 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:51.021205    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daca7a4e-29b6-425d-b8b3-0f665bdb229c-tls-certs\") pod \"noten-model-4503d2-kserve-7766d97bd7-7qffj\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") " pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:51:51.021441 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:51.021340    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-model-cache\") pod \"noten-model-4503d2-kserve-7766d97bd7-7qffj\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") " pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:51:51.021441 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:51.021387    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcm9t\" (UniqueName: \"kubernetes.io/projected/daca7a4e-29b6-425d-b8b3-0f665bdb229c-kube-api-access-lcm9t\") pod \"noten-model-4503d2-kserve-7766d97bd7-7qffj\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") " pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:51:51.021441 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:51.021424    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-home\") pod \"noten-model-4503d2-kserve-7766d97bd7-7qffj\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") " pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:51:51.021606 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:51.021486    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-tmp-dir\") pod \"noten-model-4503d2-kserve-7766d97bd7-7qffj\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") " pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:51:51.021680 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:51.021604    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-model-cache\") pod \"noten-model-4503d2-kserve-7766d97bd7-7qffj\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") " pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:51:51.023588 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:51.023566    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-dshm\") pod \"noten-model-4503d2-kserve-7766d97bd7-7qffj\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") " pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:51:51.023900 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:51.023881    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daca7a4e-29b6-425d-b8b3-0f665bdb229c-tls-certs\") pod \"noten-model-4503d2-kserve-7766d97bd7-7qffj\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") " pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:51:51.029514 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:51.029491    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcm9t\" (UniqueName: \"kubernetes.io/projected/daca7a4e-29b6-425d-b8b3-0f665bdb229c-kube-api-access-lcm9t\") pod \"noten-model-4503d2-kserve-7766d97bd7-7qffj\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") " pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:51:51.129950 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:51.129860    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:51:51.406883 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:51.406856    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"]
Sep 11 11:51:51.408186 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:51:51.408161    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddaca7a4e_29b6_425d_b8b3_0f665bdb229c.slice/crio-52e7c643376545c1ce44411c3b08aba250d7a0cfe8fdc4310c3f7e4ad4f2633a WatchSource:0}: Error finding container 52e7c643376545c1ce44411c3b08aba250d7a0cfe8fdc4310c3f7e4ad4f2633a: Status 404 returned error can't find the container with id 52e7c643376545c1ce44411c3b08aba250d7a0cfe8fdc4310c3f7e4ad4f2633a
Sep 11 11:51:52.351588 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:52.351551    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj" event={"ID":"daca7a4e-29b6-425d-b8b3-0f665bdb229c","Type":"ContainerStarted","Data":"72a36b3d5a5f218644d020015068dc56a90417e5aacf8c829ba1ad1c0f2ca33b"}
Sep 11 11:51:52.351994 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:52.351596    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj" event={"ID":"daca7a4e-29b6-425d-b8b3-0f665bdb229c","Type":"ContainerStarted","Data":"52e7c643376545c1ce44411c3b08aba250d7a0cfe8fdc4310c3f7e4ad4f2633a"}
Sep 11 11:51:52.371885 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:52.371835    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj" podStartSLOduration=2.371819269 podStartE2EDuration="2.371819269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:51:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:51:52.370838462 +0000 UTC m=+1052.791444490" watchObservedRunningTime="2026-09-11 11:51:52.371819269 +0000 UTC m=+1052.792425295"
Sep 11 11:51:53.504946 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:53.504919    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:53.572257 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:53.572225    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:53.758864 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:53.758778    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:54.078101 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:54.078012    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:55.209509 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:55.209468    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:57.527564 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:57.527530    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:58.422671 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:58.422614    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:58.634648 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:58.634601    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"]
Sep 11 11:51:58.638717 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:58.638694    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"
Sep 11 11:51:58.641198 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:58.641179    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-5745b581-serving-cert\""
Sep 11 11:51:58.652057 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:58.652032    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"]
Sep 11 11:51:58.667322 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:58.667291    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:58.696891 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:58.696808    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88bffa92-cb01-4b89-814c-5b217fd6a898-metrics-certs\") pod \"maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh\" (UID: \"88bffa92-cb01-4b89-814c-5b217fd6a898\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"
Sep 11 11:51:58.696891 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:58.696851    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6thck\" (UniqueName: \"kubernetes.io/projected/88bffa92-cb01-4b89-814c-5b217fd6a898-kube-api-access-6thck\") pod \"maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh\" (UID: \"88bffa92-cb01-4b89-814c-5b217fd6a898\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"
Sep 11 11:51:58.697087 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:58.696931    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88bffa92-cb01-4b89-814c-5b217fd6a898-maas-api-tls\") pod \"maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh\" (UID: \"88bffa92-cb01-4b89-814c-5b217fd6a898\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"
Sep 11 11:51:58.697087 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:58.697013    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88bffa92-cb01-4b89-814c-5b217fd6a898-openshift-service-ca\") pod \"maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh\" (UID: \"88bffa92-cb01-4b89-814c-5b217fd6a898\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"
Sep 11 11:51:58.797543 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:58.797509    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88bffa92-cb01-4b89-814c-5b217fd6a898-metrics-certs\") pod \"maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh\" (UID: \"88bffa92-cb01-4b89-814c-5b217fd6a898\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"
Sep 11 11:51:58.797543 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:58.797550    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6thck\" (UniqueName: \"kubernetes.io/projected/88bffa92-cb01-4b89-814c-5b217fd6a898-kube-api-access-6thck\") pod \"maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh\" (UID: \"88bffa92-cb01-4b89-814c-5b217fd6a898\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"
Sep 11 11:51:58.797779 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:58.797593    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88bffa92-cb01-4b89-814c-5b217fd6a898-maas-api-tls\") pod \"maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh\" (UID: \"88bffa92-cb01-4b89-814c-5b217fd6a898\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"
Sep 11 11:51:58.797779 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:58.797680    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88bffa92-cb01-4b89-814c-5b217fd6a898-openshift-service-ca\") pod \"maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh\" (UID: \"88bffa92-cb01-4b89-814c-5b217fd6a898\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"
Sep 11 11:51:58.797779 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:51:58.797735    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-serving-cert: secret "maas-api-e2e-mt-5745b581-serving-cert" not found
Sep 11 11:51:58.797898 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:51:58.797814    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88bffa92-cb01-4b89-814c-5b217fd6a898-maas-api-tls podName:88bffa92-cb01-4b89-814c-5b217fd6a898 nodeName:}" failed. No retries permitted until 2026-09-11 11:51:59.297796584 +0000 UTC m=+1059.718402589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/88bffa92-cb01-4b89-814c-5b217fd6a898-maas-api-tls") pod "maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh" (UID: "88bffa92-cb01-4b89-814c-5b217fd6a898") : secret "maas-api-e2e-mt-5745b581-serving-cert" not found
Sep 11 11:51:58.798325 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:58.798305    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88bffa92-cb01-4b89-814c-5b217fd6a898-openshift-service-ca\") pod \"maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh\" (UID: \"88bffa92-cb01-4b89-814c-5b217fd6a898\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"
Sep 11 11:51:58.800196 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:58.800175    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88bffa92-cb01-4b89-814c-5b217fd6a898-metrics-certs\") pod \"maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh\" (UID: \"88bffa92-cb01-4b89-814c-5b217fd6a898\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"
Sep 11 11:51:58.808877 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:58.808856    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6thck\" (UniqueName: \"kubernetes.io/projected/88bffa92-cb01-4b89-814c-5b217fd6a898-kube-api-access-6thck\") pod \"maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh\" (UID: \"88bffa92-cb01-4b89-814c-5b217fd6a898\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"
Sep 11 11:51:59.302722 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:59.302689    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88bffa92-cb01-4b89-814c-5b217fd6a898-maas-api-tls\") pod \"maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh\" (UID: \"88bffa92-cb01-4b89-814c-5b217fd6a898\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"
Sep 11 11:51:59.305165 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:59.305129    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88bffa92-cb01-4b89-814c-5b217fd6a898-maas-api-tls\") pod \"maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh\" (UID: \"88bffa92-cb01-4b89-814c-5b217fd6a898\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"
Sep 11 11:51:59.374471 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:59.374430    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:59.468678 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:59.468621    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:51:59.550165 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:59.550138    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"
Sep 11 11:51:59.840680 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:51:59.840448    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"]
Sep 11 11:51:59.843713 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:51:59.843683    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88bffa92_cb01_4b89_814c_5b217fd6a898.slice/crio-272c64f5500b2b6fd777835ac036534114b3619fc4d09da429b98f3d2d7d83e5 WatchSource:0}: Error finding container 272c64f5500b2b6fd777835ac036534114b3619fc4d09da429b98f3d2d7d83e5: Status 404 returned error can't find the container with id 272c64f5500b2b6fd777835ac036534114b3619fc4d09da429b98f3d2d7d83e5
Sep 11 11:52:00.395076 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:00.395041    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh" event={"ID":"88bffa92-cb01-4b89-814c-5b217fd6a898","Type":"ContainerStarted","Data":"272c64f5500b2b6fd777835ac036534114b3619fc4d09da429b98f3d2d7d83e5"}
Sep 11 11:52:01.131004 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:01.130972    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:52:01.131397 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:01.131019    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:52:01.144282 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:01.144254    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:52:01.402048 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:01.401950    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh" event={"ID":"88bffa92-cb01-4b89-814c-5b217fd6a898","Type":"ContainerStarted","Data":"ab9dcbc7403c0ea190530f154a00bd63dc770ad4b3b2ea5694293109dffbcda3"}
Sep 11 11:52:01.413283 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:01.413252    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:52:01.421031 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:01.420989    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh" podStartSLOduration=2.810163765 podStartE2EDuration="3.420974272s" totalImagesPullingTime="610.810507ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:52:01.420099159 +0000 UTC m=+1061.840705186" watchObservedRunningTime="2026-09-11 11:52:01.420974272 +0000 UTC m=+1061.841580299"
Sep 11 11:52:02.407591 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:02.407559    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"
Sep 11 11:52:08.417035 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:08.416996    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"
Sep 11 11:52:16.583981 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:16.583764    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"]
Sep 11 11:52:16.610719 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:16.610692    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:16.734112 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:16.734077    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:16.949993 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:16.949909    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:17.099808 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:17.099776    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:22.075561 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:22.075528    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:22.380447 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:22.380359    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:23.015219 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:23.015187    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:23.112607 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:23.112576    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:23.303594 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:23.303504    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:23.411378 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:23.411344    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:24.338836 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:24.338798    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:24.655443 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:24.655356    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:29.386782 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:29.386742    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:29.488200 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:29.488165    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:29.896671 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:29.896621    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:31.605042 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:31.604989    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj" podUID="daca7a4e-29b6-425d-b8b3-0f665bdb229c" containerName="main" containerID="cri-o://72a36b3d5a5f218644d020015068dc56a90417e5aacf8c829ba1ad1c0f2ca33b" gracePeriod=45
Sep 11 11:52:31.859892 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:31.859832    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:52:31.926384 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:31.926357    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-model-cache\") pod \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") "
Sep 11 11:52:31.926549 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:31.926397    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcm9t\" (UniqueName: \"kubernetes.io/projected/daca7a4e-29b6-425d-b8b3-0f665bdb229c-kube-api-access-lcm9t\") pod \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") "
Sep 11 11:52:31.926549 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:31.926456    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-tmp-dir\") pod \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") "
Sep 11 11:52:31.926549 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:31.926476    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-dshm\") pod \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") "
Sep 11 11:52:31.926549 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:31.926492    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daca7a4e-29b6-425d-b8b3-0f665bdb229c-tls-certs\") pod \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") "
Sep 11 11:52:31.926549 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:31.926538    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-home\") pod \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\" (UID: \"daca7a4e-29b6-425d-b8b3-0f665bdb229c\") "
Sep 11 11:52:31.926827 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:31.926690    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-model-cache" (OuterVolumeSpecName: "model-cache") pod "daca7a4e-29b6-425d-b8b3-0f665bdb229c" (UID: "daca7a4e-29b6-425d-b8b3-0f665bdb229c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:52:31.926827 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:31.926764    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "daca7a4e-29b6-425d-b8b3-0f665bdb229c" (UID: "daca7a4e-29b6-425d-b8b3-0f665bdb229c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:52:31.926928 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:31.926868    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-home" (OuterVolumeSpecName: "home") pod "daca7a4e-29b6-425d-b8b3-0f665bdb229c" (UID: "daca7a4e-29b6-425d-b8b3-0f665bdb229c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:52:31.927003 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:31.926987    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-tmp-dir\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:52:31.927042 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:31.927010    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-home\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:52:31.927042 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:31.927023    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-model-cache\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:52:31.928880 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:31.928851    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/daca7a4e-29b6-425d-b8b3-0f665bdb229c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "daca7a4e-29b6-425d-b8b3-0f665bdb229c" (UID: "daca7a4e-29b6-425d-b8b3-0f665bdb229c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:31.928983 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:31.928901    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-dshm" (OuterVolumeSpecName: "dshm") pod "daca7a4e-29b6-425d-b8b3-0f665bdb229c" (UID: "daca7a4e-29b6-425d-b8b3-0f665bdb229c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:52:31.928983 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:31.928918    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daca7a4e-29b6-425d-b8b3-0f665bdb229c-kube-api-access-lcm9t" (OuterVolumeSpecName: "kube-api-access-lcm9t") pod "daca7a4e-29b6-425d-b8b3-0f665bdb229c" (UID: "daca7a4e-29b6-425d-b8b3-0f665bdb229c"). InnerVolumeSpecName "kube-api-access-lcm9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:52:32.027677 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:32.027622    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daca7a4e-29b6-425d-b8b3-0f665bdb229c-dshm\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:52:32.027677 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:32.027671    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daca7a4e-29b6-425d-b8b3-0f665bdb229c-tls-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:52:32.027677 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:32.027682    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcm9t\" (UniqueName: \"kubernetes.io/projected/daca7a4e-29b6-425d-b8b3-0f665bdb229c-kube-api-access-lcm9t\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:52:32.564627 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:32.564596    2575 generic.go:356] "Generic (PLEG): container finished" podID="daca7a4e-29b6-425d-b8b3-0f665bdb229c" containerID="72a36b3d5a5f218644d020015068dc56a90417e5aacf8c829ba1ad1c0f2ca33b" exitCode=0
Sep 11 11:52:32.564803 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:32.564659    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"
Sep 11 11:52:32.564803 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:32.564675    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj" event={"ID":"daca7a4e-29b6-425d-b8b3-0f665bdb229c","Type":"ContainerDied","Data":"72a36b3d5a5f218644d020015068dc56a90417e5aacf8c829ba1ad1c0f2ca33b"}
Sep 11 11:52:32.564803 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:32.564724    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj" event={"ID":"daca7a4e-29b6-425d-b8b3-0f665bdb229c","Type":"ContainerDied","Data":"52e7c643376545c1ce44411c3b08aba250d7a0cfe8fdc4310c3f7e4ad4f2633a"}
Sep 11 11:52:32.564803 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:32.564746    2575 scope.go:117] "RemoveContainer" containerID="72a36b3d5a5f218644d020015068dc56a90417e5aacf8c829ba1ad1c0f2ca33b"
Sep 11 11:52:32.573771 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:32.573752    2575 scope.go:117] "RemoveContainer" containerID="72a36b3d5a5f218644d020015068dc56a90417e5aacf8c829ba1ad1c0f2ca33b"
Sep 11 11:52:32.574025 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:52:32.574009    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72a36b3d5a5f218644d020015068dc56a90417e5aacf8c829ba1ad1c0f2ca33b\": container with ID starting with 72a36b3d5a5f218644d020015068dc56a90417e5aacf8c829ba1ad1c0f2ca33b not found: ID does not exist" containerID="72a36b3d5a5f218644d020015068dc56a90417e5aacf8c829ba1ad1c0f2ca33b"
Sep 11 11:52:32.574077 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:32.574035    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72a36b3d5a5f218644d020015068dc56a90417e5aacf8c829ba1ad1c0f2ca33b"} err="failed to get container status \"72a36b3d5a5f218644d020015068dc56a90417e5aacf8c829ba1ad1c0f2ca33b\": rpc error: code = NotFound desc = could not find container \"72a36b3d5a5f218644d020015068dc56a90417e5aacf8c829ba1ad1c0f2ca33b\": container with ID starting with 72a36b3d5a5f218644d020015068dc56a90417e5aacf8c829ba1ad1c0f2ca33b not found: ID does not exist"
Sep 11 11:52:32.580053 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:32.580026    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"]
Sep 11 11:52:32.583944 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:32.583923    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-4503d2/noten-model-4503d2-kserve-7766d97bd7-7qffj"]
Sep 11 11:52:34.081355 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:34.081318    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="daca7a4e-29b6-425d-b8b3-0f665bdb229c" path="/var/lib/kubelet/pods/daca7a4e-29b6-425d-b8b3-0f665bdb229c/volumes"
Sep 11 11:52:34.564681 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:34.564649    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:35.234608 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.234577    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"]
Sep 11 11:52:35.235003 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.234796    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp" podUID="7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a" containerName="maas-api" containerID="cri-o://a16224d73daa6007c0ad8770bb48764f6bfd513cb29238f5e26b0be1ef71874a" gracePeriod=30
Sep 11 11:52:35.296821 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.296793    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98"]
Sep 11 11:52:35.297033 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.297003    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98" podUID="0993636b-9ec7-4e63-b5f0-856a883ad585" containerName="payload-processing" containerID="cri-o://872ecfcd011a27babe58b062e071dd1255515ae15d6eb9f7d863dae9d0914c6e" gracePeriod=30
Sep 11 11:52:35.445521 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.445490    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:35.543250 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.543206    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:35.548143 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.548122    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"
Sep 11 11:52:35.580043 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.580021    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98"
Sep 11 11:52:35.583363 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.583335    2575 generic.go:356] "Generic (PLEG): container finished" podID="7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a" containerID="a16224d73daa6007c0ad8770bb48764f6bfd513cb29238f5e26b0be1ef71874a" exitCode=0
Sep 11 11:52:35.583454 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.583373    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"
Sep 11 11:52:35.583524 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.583456    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp" event={"ID":"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a","Type":"ContainerDied","Data":"a16224d73daa6007c0ad8770bb48764f6bfd513cb29238f5e26b0be1ef71874a"}
Sep 11 11:52:35.583762 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.583520    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp" event={"ID":"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a","Type":"ContainerDied","Data":"a6c583df128a72b026b951235f6f0cf729b356a5f065b40f1a807a74e35c521e"}
Sep 11 11:52:35.583762 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.583553    2575 scope.go:117] "RemoveContainer" containerID="a16224d73daa6007c0ad8770bb48764f6bfd513cb29238f5e26b0be1ef71874a"
Sep 11 11:52:35.585782 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.585718    2575 generic.go:356] "Generic (PLEG): container finished" podID="0993636b-9ec7-4e63-b5f0-856a883ad585" containerID="872ecfcd011a27babe58b062e071dd1255515ae15d6eb9f7d863dae9d0914c6e" exitCode=0
Sep 11 11:52:35.585782 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.585757    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98"
Sep 11 11:52:35.585949 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.585827    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98" event={"ID":"0993636b-9ec7-4e63-b5f0-856a883ad585","Type":"ContainerDied","Data":"872ecfcd011a27babe58b062e071dd1255515ae15d6eb9f7d863dae9d0914c6e"}
Sep 11 11:52:35.585949 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.585869    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98" event={"ID":"0993636b-9ec7-4e63-b5f0-856a883ad585","Type":"ContainerDied","Data":"2509ffb4db6b59cc775663868dad619335ff573c2b650e14c3a72ec32fde2933"}
Sep 11 11:52:35.594841 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.594825    2575 scope.go:117] "RemoveContainer" containerID="a16224d73daa6007c0ad8770bb48764f6bfd513cb29238f5e26b0be1ef71874a"
Sep 11 11:52:35.595107 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:52:35.595092    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a16224d73daa6007c0ad8770bb48764f6bfd513cb29238f5e26b0be1ef71874a\": container with ID starting with a16224d73daa6007c0ad8770bb48764f6bfd513cb29238f5e26b0be1ef71874a not found: ID does not exist" containerID="a16224d73daa6007c0ad8770bb48764f6bfd513cb29238f5e26b0be1ef71874a"
Sep 11 11:52:35.595207 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.595125    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a16224d73daa6007c0ad8770bb48764f6bfd513cb29238f5e26b0be1ef71874a"} err="failed to get container status \"a16224d73daa6007c0ad8770bb48764f6bfd513cb29238f5e26b0be1ef71874a\": rpc error: code = NotFound desc = could not find container \"a16224d73daa6007c0ad8770bb48764f6bfd513cb29238f5e26b0be1ef71874a\": container with ID starting with a16224d73daa6007c0ad8770bb48764f6bfd513cb29238f5e26b0be1ef71874a not found: ID does not exist"
Sep 11 11:52:35.595207 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.595142    2575 scope.go:117] "RemoveContainer" containerID="872ecfcd011a27babe58b062e071dd1255515ae15d6eb9f7d863dae9d0914c6e"
Sep 11 11:52:35.617718 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.617690    2575 scope.go:117] "RemoveContainer" containerID="872ecfcd011a27babe58b062e071dd1255515ae15d6eb9f7d863dae9d0914c6e"
Sep 11 11:52:35.618089 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:52:35.618069    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"872ecfcd011a27babe58b062e071dd1255515ae15d6eb9f7d863dae9d0914c6e\": container with ID starting with 872ecfcd011a27babe58b062e071dd1255515ae15d6eb9f7d863dae9d0914c6e not found: ID does not exist" containerID="872ecfcd011a27babe58b062e071dd1255515ae15d6eb9f7d863dae9d0914c6e"
Sep 11 11:52:35.618161 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.618098    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"872ecfcd011a27babe58b062e071dd1255515ae15d6eb9f7d863dae9d0914c6e"} err="failed to get container status \"872ecfcd011a27babe58b062e071dd1255515ae15d6eb9f7d863dae9d0914c6e\": rpc error: code = NotFound desc = could not find container \"872ecfcd011a27babe58b062e071dd1255515ae15d6eb9f7d863dae9d0914c6e\": container with ID starting with 872ecfcd011a27babe58b062e071dd1255515ae15d6eb9f7d863dae9d0914c6e not found: ID does not exist"
Sep 11 11:52:35.666285 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.666258    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-openshift-service-ca\") pod \"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a\" (UID: \"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a\") "
Sep 11 11:52:35.666440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.666298    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0993636b-9ec7-4e63-b5f0-856a883ad585-plugins-config-volume\") pod \"0993636b-9ec7-4e63-b5f0-856a883ad585\" (UID: \"0993636b-9ec7-4e63-b5f0-856a883ad585\") "
Sep 11 11:52:35.666440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.666317    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72z99\" (UniqueName: \"kubernetes.io/projected/0993636b-9ec7-4e63-b5f0-856a883ad585-kube-api-access-72z99\") pod \"0993636b-9ec7-4e63-b5f0-856a883ad585\" (UID: \"0993636b-9ec7-4e63-b5f0-856a883ad585\") "
Sep 11 11:52:35.666440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.666368    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qllbk\" (UniqueName: \"kubernetes.io/projected/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-kube-api-access-qllbk\") pod \"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a\" (UID: \"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a\") "
Sep 11 11:52:35.666440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.666391    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-maas-api-tls\") pod \"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a\" (UID: \"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a\") "
Sep 11 11:52:35.666440 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.666435    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-metrics-certs\") pod \"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a\" (UID: \"7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a\") "
Sep 11 11:52:35.666762 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.666681    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a" (UID: "7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:52:35.666762 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.666747    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0993636b-9ec7-4e63-b5f0-856a883ad585-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0993636b-9ec7-4e63-b5f0-856a883ad585" (UID: "0993636b-9ec7-4e63-b5f0-856a883ad585"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:52:35.666859 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.666843    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-openshift-service-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:52:35.668605 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.668582    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-kube-api-access-qllbk" (OuterVolumeSpecName: "kube-api-access-qllbk") pod "7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a" (UID: "7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a"). InnerVolumeSpecName "kube-api-access-qllbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:52:35.668696 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.668601    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a" (UID: "7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:35.669048 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.669026    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a" (UID: "7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:35.669132 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.669064    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0993636b-9ec7-4e63-b5f0-856a883ad585-kube-api-access-72z99" (OuterVolumeSpecName: "kube-api-access-72z99") pod "0993636b-9ec7-4e63-b5f0-856a883ad585" (UID: "0993636b-9ec7-4e63-b5f0-856a883ad585"). InnerVolumeSpecName "kube-api-access-72z99". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:52:35.768233 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.768152    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qllbk\" (UniqueName: \"kubernetes.io/projected/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-kube-api-access-qllbk\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:52:35.768233 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.768180    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-maas-api-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:52:35.768233 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.768191    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a-metrics-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:52:35.768233 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.768198    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0993636b-9ec7-4e63-b5f0-856a883ad585-plugins-config-volume\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:52:35.768233 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.768207    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72z99\" (UniqueName: \"kubernetes.io/projected/0993636b-9ec7-4e63-b5f0-856a883ad585-kube-api-access-72z99\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:52:35.872684 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.872619    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:35.916724 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.916695    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"]
Sep 11 11:52:35.920808 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.920785    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dbe0e7-67989f7d85-5jkrp"]
Sep 11 11:52:35.936611 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.936588    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98"]
Sep 11 11:52:35.938294 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:35.938276    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-dbe0e7-d95777455-wnw98"]
Sep 11 11:52:36.081784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:36.081706    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0993636b-9ec7-4e63-b5f0-856a883ad585" path="/var/lib/kubelet/pods/0993636b-9ec7-4e63-b5f0-856a883ad585/volumes"
Sep 11 11:52:36.082107 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:36.082094    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a" path="/var/lib/kubelet/pods/7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a/volumes"
Sep 11 11:52:39.929193 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:39.929164    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:40.665266 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:40.665239    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"]
Sep 11 11:52:40.665478 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:40.665453    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh" podUID="88bffa92-cb01-4b89-814c-5b217fd6a898" containerName="maas-api" containerID="cri-o://ab9dcbc7403c0ea190530f154a00bd63dc770ad4b3b2ea5694293109dffbcda3" gracePeriod=30
Sep 11 11:52:40.689297 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:40.689263    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:40.827231 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:40.827188    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:40.921274 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:40.921209    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"
Sep 11 11:52:40.963452 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:40.963419    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:41.018936 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:41.018898    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88bffa92-cb01-4b89-814c-5b217fd6a898-maas-api-tls\") pod \"88bffa92-cb01-4b89-814c-5b217fd6a898\" (UID: \"88bffa92-cb01-4b89-814c-5b217fd6a898\") "
Sep 11 11:52:41.019300 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:41.018955    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88bffa92-cb01-4b89-814c-5b217fd6a898-openshift-service-ca\") pod \"88bffa92-cb01-4b89-814c-5b217fd6a898\" (UID: \"88bffa92-cb01-4b89-814c-5b217fd6a898\") "
Sep 11 11:52:41.019300 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:41.019057    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88bffa92-cb01-4b89-814c-5b217fd6a898-metrics-certs\") pod \"88bffa92-cb01-4b89-814c-5b217fd6a898\" (UID: \"88bffa92-cb01-4b89-814c-5b217fd6a898\") "
Sep 11 11:52:41.019300 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:41.019166    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6thck\" (UniqueName: \"kubernetes.io/projected/88bffa92-cb01-4b89-814c-5b217fd6a898-kube-api-access-6thck\") pod \"88bffa92-cb01-4b89-814c-5b217fd6a898\" (UID: \"88bffa92-cb01-4b89-814c-5b217fd6a898\") "
Sep 11 11:52:41.019666 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:41.019426    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88bffa92-cb01-4b89-814c-5b217fd6a898-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "88bffa92-cb01-4b89-814c-5b217fd6a898" (UID: "88bffa92-cb01-4b89-814c-5b217fd6a898"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:52:41.021510 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:41.021485    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88bffa92-cb01-4b89-814c-5b217fd6a898-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "88bffa92-cb01-4b89-814c-5b217fd6a898" (UID: "88bffa92-cb01-4b89-814c-5b217fd6a898"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:41.021608 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:41.021544    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88bffa92-cb01-4b89-814c-5b217fd6a898-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "88bffa92-cb01-4b89-814c-5b217fd6a898" (UID: "88bffa92-cb01-4b89-814c-5b217fd6a898"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:41.021608 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:41.021547    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88bffa92-cb01-4b89-814c-5b217fd6a898-kube-api-access-6thck" (OuterVolumeSpecName: "kube-api-access-6thck") pod "88bffa92-cb01-4b89-814c-5b217fd6a898" (UID: "88bffa92-cb01-4b89-814c-5b217fd6a898"). InnerVolumeSpecName "kube-api-access-6thck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:52:41.120987 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:41.120959    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88bffa92-cb01-4b89-814c-5b217fd6a898-metrics-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:52:41.120987 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:41.120984    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6thck\" (UniqueName: \"kubernetes.io/projected/88bffa92-cb01-4b89-814c-5b217fd6a898-kube-api-access-6thck\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:52:41.120987 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:41.120994    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88bffa92-cb01-4b89-814c-5b217fd6a898-maas-api-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:52:41.121211 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:41.121003    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88bffa92-cb01-4b89-814c-5b217fd6a898-openshift-service-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:52:41.301286 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:41.301250    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:41.397916 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:41.397877    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:41.620481 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:41.620382    2575 generic.go:356] "Generic (PLEG): container finished" podID="88bffa92-cb01-4b89-814c-5b217fd6a898" containerID="ab9dcbc7403c0ea190530f154a00bd63dc770ad4b3b2ea5694293109dffbcda3" exitCode=0
Sep 11 11:52:41.620481 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:41.620429    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"
Sep 11 11:52:41.620710 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:41.620482    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh" event={"ID":"88bffa92-cb01-4b89-814c-5b217fd6a898","Type":"ContainerDied","Data":"ab9dcbc7403c0ea190530f154a00bd63dc770ad4b3b2ea5694293109dffbcda3"}
Sep 11 11:52:41.620710 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:41.620533    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh" event={"ID":"88bffa92-cb01-4b89-814c-5b217fd6a898","Type":"ContainerDied","Data":"272c64f5500b2b6fd777835ac036534114b3619fc4d09da429b98f3d2d7d83e5"}
Sep 11 11:52:41.620710 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:41.620556    2575 scope.go:117] "RemoveContainer" containerID="ab9dcbc7403c0ea190530f154a00bd63dc770ad4b3b2ea5694293109dffbcda3"
Sep 11 11:52:41.630390 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:41.630377    2575 scope.go:117] "RemoveContainer" containerID="ab9dcbc7403c0ea190530f154a00bd63dc770ad4b3b2ea5694293109dffbcda3"
Sep 11 11:52:41.630673 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:52:41.630655    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab9dcbc7403c0ea190530f154a00bd63dc770ad4b3b2ea5694293109dffbcda3\": container with ID starting with ab9dcbc7403c0ea190530f154a00bd63dc770ad4b3b2ea5694293109dffbcda3 not found: ID does not exist" containerID="ab9dcbc7403c0ea190530f154a00bd63dc770ad4b3b2ea5694293109dffbcda3"
Sep 11 11:52:41.630726 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:41.630682    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab9dcbc7403c0ea190530f154a00bd63dc770ad4b3b2ea5694293109dffbcda3"} err="failed to get container status \"ab9dcbc7403c0ea190530f154a00bd63dc770ad4b3b2ea5694293109dffbcda3\": rpc error: code = NotFound desc = could not find container \"ab9dcbc7403c0ea190530f154a00bd63dc770ad4b3b2ea5694293109dffbcda3\": container with ID starting with ab9dcbc7403c0ea190530f154a00bd63dc770ad4b3b2ea5694293109dffbcda3 not found: ID does not exist"
Sep 11 11:52:41.642682 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:41.642660    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"]
Sep 11 11:52:41.650554 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:41.650536    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5745b581-5d4f8594c5-rp6nh"]
Sep 11 11:52:42.083070 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:42.083038    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88bffa92-cb01-4b89-814c-5b217fd6a898" path="/var/lib/kubelet/pods/88bffa92-cb01-4b89-814c-5b217fd6a898/volumes"
Sep 11 11:52:43.439657 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:43.439599    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:43.539617 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:43.539577    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:43.701970 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:43.701873    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:44.160392 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:44.160361    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:48.014160 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:48.014128    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:48.106367 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:48.106320    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:48.206220 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:48.206192    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:54.806953 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:54.806918    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"]
Sep 11 11:52:54.807399 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:54.807108    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52" podUID="464427ef-23dc-471f-a0e1-7fa6c84625bf" containerName="maas-api" containerID="cri-o://e0ee0d639eab786215d73221f204d7668d2c37651f6761ce08e4124dbdc41ebe" gracePeriod=30
Sep 11 11:52:54.852769 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:54.852731    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:54.899490 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:54.899450    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7"]
Sep 11 11:52:54.899723 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:54.899699    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7" podUID="7c233f4b-1aa2-404f-86d4-15fad1d74569" containerName="payload-processing" containerID="cri-o://42bfc11f890199e255ef3bf0481d935afb974845c93e6204ab5771ebe2df1301" gracePeriod=30
Sep 11 11:52:54.970938 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:54.970905    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:55.082558 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.082520    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:55.148751 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.148729    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"
Sep 11 11:52:55.186944 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.186902    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:55.192004 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.191981    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7"
Sep 11 11:52:55.263614 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.263580    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/464427ef-23dc-471f-a0e1-7fa6c84625bf-openshift-service-ca\") pod \"464427ef-23dc-471f-a0e1-7fa6c84625bf\" (UID: \"464427ef-23dc-471f-a0e1-7fa6c84625bf\") "
Sep 11 11:52:55.263614 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.263618    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c233f4b-1aa2-404f-86d4-15fad1d74569-plugins-config-volume\") pod \"7c233f4b-1aa2-404f-86d4-15fad1d74569\" (UID: \"7c233f4b-1aa2-404f-86d4-15fad1d74569\") "
Sep 11 11:52:55.263885 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.263699    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5dp4\" (UniqueName: \"kubernetes.io/projected/7c233f4b-1aa2-404f-86d4-15fad1d74569-kube-api-access-z5dp4\") pod \"7c233f4b-1aa2-404f-86d4-15fad1d74569\" (UID: \"7c233f4b-1aa2-404f-86d4-15fad1d74569\") "
Sep 11 11:52:55.263885 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.263747    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6264\" (UniqueName: \"kubernetes.io/projected/464427ef-23dc-471f-a0e1-7fa6c84625bf-kube-api-access-g6264\") pod \"464427ef-23dc-471f-a0e1-7fa6c84625bf\" (UID: \"464427ef-23dc-471f-a0e1-7fa6c84625bf\") "
Sep 11 11:52:55.263885 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.263784    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/464427ef-23dc-471f-a0e1-7fa6c84625bf-metrics-certs\") pod \"464427ef-23dc-471f-a0e1-7fa6c84625bf\" (UID: \"464427ef-23dc-471f-a0e1-7fa6c84625bf\") "
Sep 11 11:52:55.263885 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.263819    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/464427ef-23dc-471f-a0e1-7fa6c84625bf-maas-api-tls\") pod \"464427ef-23dc-471f-a0e1-7fa6c84625bf\" (UID: \"464427ef-23dc-471f-a0e1-7fa6c84625bf\") "
Sep 11 11:52:55.264115 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.263977    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/464427ef-23dc-471f-a0e1-7fa6c84625bf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "464427ef-23dc-471f-a0e1-7fa6c84625bf" (UID: "464427ef-23dc-471f-a0e1-7fa6c84625bf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:52:55.264169 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.264117    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c233f4b-1aa2-404f-86d4-15fad1d74569-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7c233f4b-1aa2-404f-86d4-15fad1d74569" (UID: "7c233f4b-1aa2-404f-86d4-15fad1d74569"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:52:55.264169 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.264154    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/464427ef-23dc-471f-a0e1-7fa6c84625bf-openshift-service-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:52:55.266163 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.266135    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/464427ef-23dc-471f-a0e1-7fa6c84625bf-kube-api-access-g6264" (OuterVolumeSpecName: "kube-api-access-g6264") pod "464427ef-23dc-471f-a0e1-7fa6c84625bf" (UID: "464427ef-23dc-471f-a0e1-7fa6c84625bf"). InnerVolumeSpecName "kube-api-access-g6264". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:52:55.266163 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.266156    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/464427ef-23dc-471f-a0e1-7fa6c84625bf-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "464427ef-23dc-471f-a0e1-7fa6c84625bf" (UID: "464427ef-23dc-471f-a0e1-7fa6c84625bf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:55.266312 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.266169    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c233f4b-1aa2-404f-86d4-15fad1d74569-kube-api-access-z5dp4" (OuterVolumeSpecName: "kube-api-access-z5dp4") pod "7c233f4b-1aa2-404f-86d4-15fad1d74569" (UID: "7c233f4b-1aa2-404f-86d4-15fad1d74569"). InnerVolumeSpecName "kube-api-access-z5dp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:52:55.266312 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.266187    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/464427ef-23dc-471f-a0e1-7fa6c84625bf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "464427ef-23dc-471f-a0e1-7fa6c84625bf" (UID: "464427ef-23dc-471f-a0e1-7fa6c84625bf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:52:55.286535 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.286514    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:55.365424 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.365342    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6264\" (UniqueName: \"kubernetes.io/projected/464427ef-23dc-471f-a0e1-7fa6c84625bf-kube-api-access-g6264\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:52:55.365424 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.365370    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/464427ef-23dc-471f-a0e1-7fa6c84625bf-metrics-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:52:55.365424 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.365383    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/464427ef-23dc-471f-a0e1-7fa6c84625bf-maas-api-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:52:55.365424 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.365392    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c233f4b-1aa2-404f-86d4-15fad1d74569-plugins-config-volume\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:52:55.365424 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.365406    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5dp4\" (UniqueName: \"kubernetes.io/projected/7c233f4b-1aa2-404f-86d4-15fad1d74569-kube-api-access-z5dp4\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:52:55.697301 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.697214    2575 generic.go:356] "Generic (PLEG): container finished" podID="464427ef-23dc-471f-a0e1-7fa6c84625bf" containerID="e0ee0d639eab786215d73221f204d7668d2c37651f6761ce08e4124dbdc41ebe" exitCode=0
Sep 11 11:52:55.697301 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.697259    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"
Sep 11 11:52:55.697546 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.697310    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52" event={"ID":"464427ef-23dc-471f-a0e1-7fa6c84625bf","Type":"ContainerDied","Data":"e0ee0d639eab786215d73221f204d7668d2c37651f6761ce08e4124dbdc41ebe"}
Sep 11 11:52:55.697546 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.697367    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52" event={"ID":"464427ef-23dc-471f-a0e1-7fa6c84625bf","Type":"ContainerDied","Data":"4c2c985d6ae0b0199533cb2366350599ce9b0efa1d63abf5ab28fd60ca19ba79"}
Sep 11 11:52:55.697546 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.697399    2575 scope.go:117] "RemoveContainer" containerID="e0ee0d639eab786215d73221f204d7668d2c37651f6761ce08e4124dbdc41ebe"
Sep 11 11:52:55.698820 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.698801    2575 generic.go:356] "Generic (PLEG): container finished" podID="7c233f4b-1aa2-404f-86d4-15fad1d74569" containerID="42bfc11f890199e255ef3bf0481d935afb974845c93e6204ab5771ebe2df1301" exitCode=0
Sep 11 11:52:55.698908 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.698837    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7" event={"ID":"7c233f4b-1aa2-404f-86d4-15fad1d74569","Type":"ContainerDied","Data":"42bfc11f890199e255ef3bf0481d935afb974845c93e6204ab5771ebe2df1301"}
Sep 11 11:52:55.698908 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.698853    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7"
Sep 11 11:52:55.698908 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.698887    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7" event={"ID":"7c233f4b-1aa2-404f-86d4-15fad1d74569","Type":"ContainerDied","Data":"b53bb1c26eb3533ee5f3b54fd515cc07cb3e3c755cb47efc447f3bf92b4fa088"}
Sep 11 11:52:55.706969 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.706950    2575 scope.go:117] "RemoveContainer" containerID="e0ee0d639eab786215d73221f204d7668d2c37651f6761ce08e4124dbdc41ebe"
Sep 11 11:52:55.707395 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:52:55.707251    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0ee0d639eab786215d73221f204d7668d2c37651f6761ce08e4124dbdc41ebe\": container with ID starting with e0ee0d639eab786215d73221f204d7668d2c37651f6761ce08e4124dbdc41ebe not found: ID does not exist" containerID="e0ee0d639eab786215d73221f204d7668d2c37651f6761ce08e4124dbdc41ebe"
Sep 11 11:52:55.707395 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.707277    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0ee0d639eab786215d73221f204d7668d2c37651f6761ce08e4124dbdc41ebe"} err="failed to get container status \"e0ee0d639eab786215d73221f204d7668d2c37651f6761ce08e4124dbdc41ebe\": rpc error: code = NotFound desc = could not find container \"e0ee0d639eab786215d73221f204d7668d2c37651f6761ce08e4124dbdc41ebe\": container with ID starting with e0ee0d639eab786215d73221f204d7668d2c37651f6761ce08e4124dbdc41ebe not found: ID does not exist"
Sep 11 11:52:55.707395 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.707292    2575 scope.go:117] "RemoveContainer" containerID="42bfc11f890199e255ef3bf0481d935afb974845c93e6204ab5771ebe2df1301"
Sep 11 11:52:55.715548 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.715531    2575 scope.go:117] "RemoveContainer" containerID="42bfc11f890199e255ef3bf0481d935afb974845c93e6204ab5771ebe2df1301"
Sep 11 11:52:55.715852 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:52:55.715833    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42bfc11f890199e255ef3bf0481d935afb974845c93e6204ab5771ebe2df1301\": container with ID starting with 42bfc11f890199e255ef3bf0481d935afb974845c93e6204ab5771ebe2df1301 not found: ID does not exist" containerID="42bfc11f890199e255ef3bf0481d935afb974845c93e6204ab5771ebe2df1301"
Sep 11 11:52:55.715893 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.715859    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42bfc11f890199e255ef3bf0481d935afb974845c93e6204ab5771ebe2df1301"} err="failed to get container status \"42bfc11f890199e255ef3bf0481d935afb974845c93e6204ab5771ebe2df1301\": rpc error: code = NotFound desc = could not find container \"42bfc11f890199e255ef3bf0481d935afb974845c93e6204ab5771ebe2df1301\": container with ID starting with 42bfc11f890199e255ef3bf0481d935afb974845c93e6204ab5771ebe2df1301 not found: ID does not exist"
Sep 11 11:52:55.729330 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.729305    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"]
Sep 11 11:52:55.734868 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.734847    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c6bcf5-8646ff976d-8ln52"]
Sep 11 11:52:55.768563 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.768490    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7"]
Sep 11 11:52:55.770503 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:55.770478    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c6bcf5-647449c96f-jcsh7"]
Sep 11 11:52:56.083002 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:56.082964    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="464427ef-23dc-471f-a0e1-7fa6c84625bf" path="/var/lib/kubelet/pods/464427ef-23dc-471f-a0e1-7fa6c84625bf/volumes"
Sep 11 11:52:56.083349 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:56.083308    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c233f4b-1aa2-404f-86d4-15fad1d74569" path="/var/lib/kubelet/pods/7c233f4b-1aa2-404f-86d4-15fad1d74569/volumes"
Sep 11 11:52:56.171956 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:56.171920    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:56.267070 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:56.267040    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:52:56.661607 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:52:56.661573    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:00.059999 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:00.059969    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:02.700235 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:02.700197    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:02.801242 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:02.801204    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:03.006069 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:03.006036    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:03.579378 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:03.579343    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:04.293806 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:04.293774    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:04.462168 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:04.462139    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:04.589600 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:04.589518    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:04.818882 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:04.818852    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:05.465475 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:05.465440    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:05.857401 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:05.857365    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:06.176610 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:06.176529    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:07.078295 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:07.078260    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:07.275300 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:07.275252    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:08.364267 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.364235    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"]
Sep 11 11:53:08.364654 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.364610    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c233f4b-1aa2-404f-86d4-15fad1d74569" containerName="payload-processing"
Sep 11 11:53:08.364654 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.364621    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c233f4b-1aa2-404f-86d4-15fad1d74569" containerName="payload-processing"
Sep 11 11:53:08.364743 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.364656    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a" containerName="maas-api"
Sep 11 11:53:08.364743 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.364662    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a" containerName="maas-api"
Sep 11 11:53:08.364743 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.364673    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0993636b-9ec7-4e63-b5f0-856a883ad585" containerName="payload-processing"
Sep 11 11:53:08.364743 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.364678    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0993636b-9ec7-4e63-b5f0-856a883ad585" containerName="payload-processing"
Sep 11 11:53:08.364743 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.364684    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daca7a4e-29b6-425d-b8b3-0f665bdb229c" containerName="main"
Sep 11 11:53:08.364743 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.364690    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="daca7a4e-29b6-425d-b8b3-0f665bdb229c" containerName="main"
Sep 11 11:53:08.364743 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.364698    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88bffa92-cb01-4b89-814c-5b217fd6a898" containerName="maas-api"
Sep 11 11:53:08.364743 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.364704    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="88bffa92-cb01-4b89-814c-5b217fd6a898" containerName="maas-api"
Sep 11 11:53:08.364743 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.364709    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="464427ef-23dc-471f-a0e1-7fa6c84625bf" containerName="maas-api"
Sep 11 11:53:08.364743 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.364714    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="464427ef-23dc-471f-a0e1-7fa6c84625bf" containerName="maas-api"
Sep 11 11:53:08.365061 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.364791    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c2fb2d3-5e2f-4a61-82e8-baa4b1ccb96a" containerName="maas-api"
Sep 11 11:53:08.365061 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.364800    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="88bffa92-cb01-4b89-814c-5b217fd6a898" containerName="maas-api"
Sep 11 11:53:08.365061 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.364807    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c233f4b-1aa2-404f-86d4-15fad1d74569" containerName="payload-processing"
Sep 11 11:53:08.365061 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.364813    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="464427ef-23dc-471f-a0e1-7fa6c84625bf" containerName="maas-api"
Sep 11 11:53:08.365061 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.364820    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0993636b-9ec7-4e63-b5f0-856a883ad585" containerName="payload-processing"
Sep 11 11:53:08.365061 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.364826    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="daca7a4e-29b6-425d-b8b3-0f665bdb229c" containerName="main"
Sep 11 11:53:08.369552 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.369527    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"
Sep 11 11:53:08.372544 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.372521    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-272c5e-serving-cert\""
Sep 11 11:53:08.407316 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.407217    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"]
Sep 11 11:53:08.441590 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.441554    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:08.487539 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.487507    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb11417d-cbd4-47cb-9449-a2df504de3e1-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk\" (UID: \"bb11417d-cbd4-47cb-9449-a2df504de3e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"
Sep 11 11:53:08.487711 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.487543    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj2qj\" (UniqueName: \"kubernetes.io/projected/bb11417d-cbd4-47cb-9449-a2df504de3e1-kube-api-access-vj2qj\") pod \"maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk\" (UID: \"bb11417d-cbd4-47cb-9449-a2df504de3e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"
Sep 11 11:53:08.487711 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.487575    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb11417d-cbd4-47cb-9449-a2df504de3e1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk\" (UID: \"bb11417d-cbd4-47cb-9449-a2df504de3e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"
Sep 11 11:53:08.487711 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.487698    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb11417d-cbd4-47cb-9449-a2df504de3e1-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk\" (UID: \"bb11417d-cbd4-47cb-9449-a2df504de3e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"
Sep 11 11:53:08.588820 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.588784    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb11417d-cbd4-47cb-9449-a2df504de3e1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk\" (UID: \"bb11417d-cbd4-47cb-9449-a2df504de3e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"
Sep 11 11:53:08.589013 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.588843    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb11417d-cbd4-47cb-9449-a2df504de3e1-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk\" (UID: \"bb11417d-cbd4-47cb-9449-a2df504de3e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"
Sep 11 11:53:08.589013 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.588933    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb11417d-cbd4-47cb-9449-a2df504de3e1-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk\" (UID: \"bb11417d-cbd4-47cb-9449-a2df504de3e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"
Sep 11 11:53:08.589013 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.588956    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj2qj\" (UniqueName: \"kubernetes.io/projected/bb11417d-cbd4-47cb-9449-a2df504de3e1-kube-api-access-vj2qj\") pod \"maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk\" (UID: \"bb11417d-cbd4-47cb-9449-a2df504de3e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"
Sep 11 11:53:08.589549 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.589515    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb11417d-cbd4-47cb-9449-a2df504de3e1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk\" (UID: \"bb11417d-cbd4-47cb-9449-a2df504de3e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"
Sep 11 11:53:08.591536 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.591508    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb11417d-cbd4-47cb-9449-a2df504de3e1-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk\" (UID: \"bb11417d-cbd4-47cb-9449-a2df504de3e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"
Sep 11 11:53:08.591687 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.591544    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb11417d-cbd4-47cb-9449-a2df504de3e1-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk\" (UID: \"bb11417d-cbd4-47cb-9449-a2df504de3e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"
Sep 11 11:53:08.602262 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.602234    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj2qj\" (UniqueName: \"kubernetes.io/projected/bb11417d-cbd4-47cb-9449-a2df504de3e1-kube-api-access-vj2qj\") pod \"maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk\" (UID: \"bb11417d-cbd4-47cb-9449-a2df504de3e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"
Sep 11 11:53:08.624584 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.624517    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:08.680724 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.680698    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"
Sep 11 11:53:08.756264 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.756216    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:08.967910 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:08.967830    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"]
Sep 11 11:53:08.970198 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:53:08.970175    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb11417d_cbd4_47cb_9449_a2df504de3e1.slice/crio-aa2813e7f88f4ac4ca2934c5eaf4b36de80504da07f2c0cafce963d5b5651165 WatchSource:0}: Error finding container aa2813e7f88f4ac4ca2934c5eaf4b36de80504da07f2c0cafce963d5b5651165: Status 404 returned error can't find the container with id aa2813e7f88f4ac4ca2934c5eaf4b36de80504da07f2c0cafce963d5b5651165
Sep 11 11:53:09.163700 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.163599    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:09.292436 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.292397    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:09.367278 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.367249    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw"]
Sep 11 11:53:09.371962 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.371941    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw"
Sep 11 11:53:09.375295 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.375276    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-272c5e-dockercfg-wstnl\""
Sep 11 11:53:09.375770 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.375749    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-272c5e\""
Sep 11 11:53:09.382653 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.382611    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn"]
Sep 11 11:53:09.390118 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.390091    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn"
Sep 11 11:53:09.391189 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.391168    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw"]
Sep 11 11:53:09.413178 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.413154    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn"]
Sep 11 11:53:09.497643 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.497599    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b9fc270-59db-4dc0-9d41-b31866b9947f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn\" (UID: \"1b9fc270-59db-4dc0-9d41-b31866b9947f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn"
Sep 11 11:53:09.497774 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.497701    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqbzj\" (UniqueName: \"kubernetes.io/projected/5dc8042b-4520-4a85-b260-d3a7ba649f6b-kube-api-access-dqbzj\") pod \"payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw\" (UID: \"5dc8042b-4520-4a85-b260-d3a7ba649f6b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw"
Sep 11 11:53:09.497826 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.497774    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-756zz\" (UniqueName: \"kubernetes.io/projected/1b9fc270-59db-4dc0-9d41-b31866b9947f-kube-api-access-756zz\") pod \"payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn\" (UID: \"1b9fc270-59db-4dc0-9d41-b31866b9947f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn"
Sep 11 11:53:09.497826 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.497801    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5dc8042b-4520-4a85-b260-d3a7ba649f6b-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw\" (UID: \"5dc8042b-4520-4a85-b260-d3a7ba649f6b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw"
Sep 11 11:53:09.599340 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.599240    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-756zz\" (UniqueName: \"kubernetes.io/projected/1b9fc270-59db-4dc0-9d41-b31866b9947f-kube-api-access-756zz\") pod \"payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn\" (UID: \"1b9fc270-59db-4dc0-9d41-b31866b9947f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn"
Sep 11 11:53:09.599340 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.599282    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5dc8042b-4520-4a85-b260-d3a7ba649f6b-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw\" (UID: \"5dc8042b-4520-4a85-b260-d3a7ba649f6b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw"
Sep 11 11:53:09.599644 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.599347    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b9fc270-59db-4dc0-9d41-b31866b9947f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn\" (UID: \"1b9fc270-59db-4dc0-9d41-b31866b9947f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn"
Sep 11 11:53:09.599644 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.599454    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqbzj\" (UniqueName: \"kubernetes.io/projected/5dc8042b-4520-4a85-b260-d3a7ba649f6b-kube-api-access-dqbzj\") pod \"payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw\" (UID: \"5dc8042b-4520-4a85-b260-d3a7ba649f6b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw"
Sep 11 11:53:09.600379 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.600357    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b9fc270-59db-4dc0-9d41-b31866b9947f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn\" (UID: \"1b9fc270-59db-4dc0-9d41-b31866b9947f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn"
Sep 11 11:53:09.600705 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.600685    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5dc8042b-4520-4a85-b260-d3a7ba649f6b-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw\" (UID: \"5dc8042b-4520-4a85-b260-d3a7ba649f6b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw"
Sep 11 11:53:09.614129 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.614104    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-756zz\" (UniqueName: \"kubernetes.io/projected/1b9fc270-59db-4dc0-9d41-b31866b9947f-kube-api-access-756zz\") pod \"payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn\" (UID: \"1b9fc270-59db-4dc0-9d41-b31866b9947f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn"
Sep 11 11:53:09.615192 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.615169    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqbzj\" (UniqueName: \"kubernetes.io/projected/5dc8042b-4520-4a85-b260-d3a7ba649f6b-kube-api-access-dqbzj\") pod \"payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw\" (UID: \"5dc8042b-4520-4a85-b260-d3a7ba649f6b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw"
Sep 11 11:53:09.683176 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.683148    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw"
Sep 11 11:53:09.701082 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.701040    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn"
Sep 11 11:53:09.778898 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.778837    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk" event={"ID":"bb11417d-cbd4-47cb-9449-a2df504de3e1","Type":"ContainerStarted","Data":"4aab7b8af96ca1af2eeea4ed84833b8db6c5634d13a081a0b32953b8fc34c1cc"}
Sep 11 11:53:09.779017 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.778944    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk" event={"ID":"bb11417d-cbd4-47cb-9449-a2df504de3e1","Type":"ContainerStarted","Data":"aa2813e7f88f4ac4ca2934c5eaf4b36de80504da07f2c0cafce963d5b5651165"}
Sep 11 11:53:09.779112 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.779096    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"
Sep 11 11:53:09.800805 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.800754    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk" podStartSLOduration=1.353922319 podStartE2EDuration="1.800737353s" totalImagesPullingTime="446.815034ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:53:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:09.798880596 +0000 UTC m=+1130.219486624" watchObservedRunningTime="2026-09-11 11:53:09.800737353 +0000 UTC m=+1130.221343425"
Sep 11 11:53:09.975893 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:09.975869    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw"]
Sep 11 11:53:09.977996 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:53:09.977964    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dc8042b_4520_4a85_b260_d3a7ba649f6b.slice/crio-229279aca31a185d7b105e31a57d1f03b52c0ce242854b5d4ec69edcfca04d8e WatchSource:0}: Error finding container 229279aca31a185d7b105e31a57d1f03b52c0ce242854b5d4ec69edcfca04d8e: Status 404 returned error can't find the container with id 229279aca31a185d7b105e31a57d1f03b52c0ce242854b5d4ec69edcfca04d8e
Sep 11 11:53:10.017984 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.017958    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn"]
Sep 11 11:53:10.024777 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:53:10.024760    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b9fc270_59db_4dc0_9d41_b31866b9947f.slice/crio-673834506bc64c157ba63f4f09cd2115cde5a00c5bdf59ea45f8b8308e19c58c WatchSource:0}: Error finding container 673834506bc64c157ba63f4f09cd2115cde5a00c5bdf59ea45f8b8308e19c58c: Status 404 returned error can't find the container with id 673834506bc64c157ba63f4f09cd2115cde5a00c5bdf59ea45f8b8308e19c58c
Sep 11 11:53:10.587185 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.587157    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"]
Sep 11 11:53:10.591077 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.591056    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"
Sep 11 11:53:10.593839 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.593821    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-8989e8-serving-cert\""
Sep 11 11:53:10.607267 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.607221    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"]
Sep 11 11:53:10.689892 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.689857    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:10.710032 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.709995    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfzdl\" (UniqueName: \"kubernetes.io/projected/ffd6ad89-a551-4650-bdf0-5a49ab808479-kube-api-access-kfzdl\") pod \"maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj\" (UID: \"ffd6ad89-a551-4650-bdf0-5a49ab808479\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"
Sep 11 11:53:10.710219 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.710135    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffd6ad89-a551-4650-bdf0-5a49ab808479-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj\" (UID: \"ffd6ad89-a551-4650-bdf0-5a49ab808479\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"
Sep 11 11:53:10.710219 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.710170    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffd6ad89-a551-4650-bdf0-5a49ab808479-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj\" (UID: \"ffd6ad89-a551-4650-bdf0-5a49ab808479\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"
Sep 11 11:53:10.710219 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.710208    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffd6ad89-a551-4650-bdf0-5a49ab808479-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj\" (UID: \"ffd6ad89-a551-4650-bdf0-5a49ab808479\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"
Sep 11 11:53:10.787052 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.787015    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw" event={"ID":"5dc8042b-4520-4a85-b260-d3a7ba649f6b","Type":"ContainerStarted","Data":"65e8507cb992d29b6f00a3ccc99ca637d5492aeb702d9189314ee596606446c0"}
Sep 11 11:53:10.787229 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.787076    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw" event={"ID":"5dc8042b-4520-4a85-b260-d3a7ba649f6b","Type":"ContainerStarted","Data":"229279aca31a185d7b105e31a57d1f03b52c0ce242854b5d4ec69edcfca04d8e"}
Sep 11 11:53:10.787229 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.787137    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw"
Sep 11 11:53:10.788783 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.788751    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn" event={"ID":"1b9fc270-59db-4dc0-9d41-b31866b9947f","Type":"ContainerStarted","Data":"7d690d1ae16ee0370eec1ba8ee529ac2218a38cbc71533e0b68fe2a8f24bd246"}
Sep 11 11:53:10.788888 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.788799    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn" event={"ID":"1b9fc270-59db-4dc0-9d41-b31866b9947f","Type":"ContainerStarted","Data":"673834506bc64c157ba63f4f09cd2115cde5a00c5bdf59ea45f8b8308e19c58c"}
Sep 11 11:53:10.788980 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.788961    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn"
Sep 11 11:53:10.811393 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.811364    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffd6ad89-a551-4650-bdf0-5a49ab808479-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj\" (UID: \"ffd6ad89-a551-4650-bdf0-5a49ab808479\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"
Sep 11 11:53:10.811510 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.811402    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffd6ad89-a551-4650-bdf0-5a49ab808479-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj\" (UID: \"ffd6ad89-a551-4650-bdf0-5a49ab808479\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"
Sep 11 11:53:10.811510 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.811450    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffd6ad89-a551-4650-bdf0-5a49ab808479-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj\" (UID: \"ffd6ad89-a551-4650-bdf0-5a49ab808479\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"
Sep 11 11:53:10.811510 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.811487    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfzdl\" (UniqueName: \"kubernetes.io/projected/ffd6ad89-a551-4650-bdf0-5a49ab808479-kube-api-access-kfzdl\") pod \"maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj\" (UID: \"ffd6ad89-a551-4650-bdf0-5a49ab808479\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"
Sep 11 11:53:10.812303 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.812278    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffd6ad89-a551-4650-bdf0-5a49ab808479-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj\" (UID: \"ffd6ad89-a551-4650-bdf0-5a49ab808479\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"
Sep 11 11:53:10.814070 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.814042    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffd6ad89-a551-4650-bdf0-5a49ab808479-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj\" (UID: \"ffd6ad89-a551-4650-bdf0-5a49ab808479\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"
Sep 11 11:53:10.814513 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.814494    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffd6ad89-a551-4650-bdf0-5a49ab808479-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj\" (UID: \"ffd6ad89-a551-4650-bdf0-5a49ab808479\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"
Sep 11 11:53:10.816409 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.816388    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:10.827365 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.827315    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw" podStartSLOduration=1.827301904 podStartE2EDuration="1.827301904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:53:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:10.825605887 +0000 UTC m=+1131.246211917" watchObservedRunningTime="2026-09-11 11:53:10.827301904 +0000 UTC m=+1131.247907931"
Sep 11 11:53:10.840299 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.840233    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfzdl\" (UniqueName: \"kubernetes.io/projected/ffd6ad89-a551-4650-bdf0-5a49ab808479-kube-api-access-kfzdl\") pod \"maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj\" (UID: \"ffd6ad89-a551-4650-bdf0-5a49ab808479\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"
Sep 11 11:53:10.871076 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.871029    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn" podStartSLOduration=1.871014222 podStartE2EDuration="1.871014222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:53:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:10.868831284 +0000 UTC m=+1131.289437311" watchObservedRunningTime="2026-09-11 11:53:10.871014222 +0000 UTC m=+1131.291620249"
Sep 11 11:53:10.925092 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:10.925057    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"
Sep 11 11:53:11.230697 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:11.230676    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"]
Sep 11 11:53:11.232069 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:53:11.232039    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffd6ad89_a551_4650_bdf0_5a49ab808479.slice/crio-d8494128069ef3ae5598d248bbdd473fbb6be3c12d6a9680d0a9e1ea08cb206b WatchSource:0}: Error finding container d8494128069ef3ae5598d248bbdd473fbb6be3c12d6a9680d0a9e1ea08cb206b: Status 404 returned error can't find the container with id d8494128069ef3ae5598d248bbdd473fbb6be3c12d6a9680d0a9e1ea08cb206b
Sep 11 11:53:11.539836 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:11.539804    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:11.732595 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:11.732551    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:11.796867 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:11.796769    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj" event={"ID":"ffd6ad89-a551-4650-bdf0-5a49ab808479","Type":"ContainerStarted","Data":"f8b38c6ad68c0d3003438b8703dfe043a71109a1cc653d71a6d51a9f3a0be409"}
Sep 11 11:53:11.796867 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:11.796832    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj" event={"ID":"ffd6ad89-a551-4650-bdf0-5a49ab808479","Type":"ContainerStarted","Data":"d8494128069ef3ae5598d248bbdd473fbb6be3c12d6a9680d0a9e1ea08cb206b"}
Sep 11 11:53:11.902243 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:11.902182    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj" podStartSLOduration=1.552781126 podStartE2EDuration="1.902158627s" totalImagesPullingTime="349.377501ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:53:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:11.899113446 +0000 UTC m=+1132.319719475" watchObservedRunningTime="2026-09-11 11:53:11.902158627 +0000 UTC m=+1132.322764653"
Sep 11 11:53:11.941787 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:11.941749    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8"]
Sep 11 11:53:11.946241 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:11.946211    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8"
Sep 11 11:53:11.971315 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:11.971284    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-8989e8\""
Sep 11 11:53:11.997121 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:11.997083    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-8989e8-dockercfg-vmcgk\""
Sep 11 11:53:12.006978 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:12.006949    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8"]
Sep 11 11:53:12.023565 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:12.023535    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a09969a-0d59-4774-a095-73b6f81c038c-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8\" (UID: \"8a09969a-0d59-4774-a095-73b6f81c038c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8"
Sep 11 11:53:12.023804 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:12.023621    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxgvj\" (UniqueName: \"kubernetes.io/projected/8a09969a-0d59-4774-a095-73b6f81c038c-kube-api-access-vxgvj\") pod \"payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8\" (UID: \"8a09969a-0d59-4774-a095-73b6f81c038c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8"
Sep 11 11:53:12.124821 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:12.124723    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a09969a-0d59-4774-a095-73b6f81c038c-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8\" (UID: \"8a09969a-0d59-4774-a095-73b6f81c038c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8"
Sep 11 11:53:12.124821 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:12.124798    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxgvj\" (UniqueName: \"kubernetes.io/projected/8a09969a-0d59-4774-a095-73b6f81c038c-kube-api-access-vxgvj\") pod \"payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8\" (UID: \"8a09969a-0d59-4774-a095-73b6f81c038c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8"
Sep 11 11:53:12.125479 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:12.125453    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a09969a-0d59-4774-a095-73b6f81c038c-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8\" (UID: \"8a09969a-0d59-4774-a095-73b6f81c038c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8"
Sep 11 11:53:12.142437 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:12.142411    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxgvj\" (UniqueName: \"kubernetes.io/projected/8a09969a-0d59-4774-a095-73b6f81c038c-kube-api-access-vxgvj\") pod \"payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8\" (UID: \"8a09969a-0d59-4774-a095-73b6f81c038c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8"
Sep 11 11:53:12.261987 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:12.261950    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8"
Sep 11 11:53:12.556271 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:12.556236    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8"]
Sep 11 11:53:12.560005 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:53:12.559980    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a09969a_0d59_4774_a095_73b6f81c038c.slice/crio-6ffbf5b8b7f50b60fb7e5b0b91c4f78d57e9c7311af3f4b2f0d4a8ea2d5fa010 WatchSource:0}: Error finding container 6ffbf5b8b7f50b60fb7e5b0b91c4f78d57e9c7311af3f4b2f0d4a8ea2d5fa010: Status 404 returned error can't find the container with id 6ffbf5b8b7f50b60fb7e5b0b91c4f78d57e9c7311af3f4b2f0d4a8ea2d5fa010
Sep 11 11:53:12.805108 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:12.805069    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8" event={"ID":"8a09969a-0d59-4774-a095-73b6f81c038c","Type":"ContainerStarted","Data":"78de7cf0edbebba4a32e6b5ef6b466e02f999e3f64fb0e5ffcd0ffa023c4b408"}
Sep 11 11:53:12.805582 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:12.805143    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8" event={"ID":"8a09969a-0d59-4774-a095-73b6f81c038c","Type":"ContainerStarted","Data":"6ffbf5b8b7f50b60fb7e5b0b91c4f78d57e9c7311af3f4b2f0d4a8ea2d5fa010"}
Sep 11 11:53:12.805582 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:12.805230    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"
Sep 11 11:53:12.857758 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:12.857613    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8" podStartSLOduration=1.8575903729999999 podStartE2EDuration="1.857590373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:53:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:12.855732975 +0000 UTC m=+1133.276339007" watchObservedRunningTime="2026-09-11 11:53:12.857590373 +0000 UTC m=+1133.278196400"
Sep 11 11:53:13.810550 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:13.810521    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8"
Sep 11 11:53:15.794948 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:15.794920    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"
Sep 11 11:53:18.816091 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:18.816055    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"
Sep 11 11:53:21.799598 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:21.799569    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw"
Sep 11 11:53:21.799990 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:21.799668    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn"
Sep 11 11:53:21.959113 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:21.959083    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:24.817816 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:24.817788    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8"
Sep 11 11:53:29.853901 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:29.853866    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:31.895322 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:31.895281    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:32.210254 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:32.210169    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:32.643653 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:32.643603    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:32.869260 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:32.869227    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:34.326479 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:34.326450    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:35.471610 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:35.471566    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:35.575586 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:35.575555    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:37.474496 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:37.474458    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:37.594118 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:37.594089    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:37.892821 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:37.892789    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:38.326295 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:38.326263    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:38.711962 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:38.711875    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:39.177197 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:39.177160    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:39.480870 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:39.480787    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:39.845014 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:39.844982    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:40.567727 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:40.567682    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:43.598800 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:43.598712    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:43.701389 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:43.701347    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:44.744297 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:44.744265    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"]
Sep 11 11:53:44.749394 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:44.749375    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"
Sep 11 11:53:44.752029 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:44.752008    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-6daa53-serving-cert\""
Sep 11 11:53:44.762257 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:44.762230    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"]
Sep 11 11:53:44.794652 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:44.794597    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:44.840339 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:44.840307    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vpz8\" (UniqueName: \"kubernetes.io/projected/edf26460-8713-439a-bd02-f26f2f81f554-kube-api-access-6vpz8\") pod \"maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft\" (UID: \"edf26460-8713-439a-bd02-f26f2f81f554\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"
Sep 11 11:53:44.840493 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:44.840412    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edf26460-8713-439a-bd02-f26f2f81f554-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft\" (UID: \"edf26460-8713-439a-bd02-f26f2f81f554\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"
Sep 11 11:53:44.840493 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:44.840444    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edf26460-8713-439a-bd02-f26f2f81f554-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft\" (UID: \"edf26460-8713-439a-bd02-f26f2f81f554\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"
Sep 11 11:53:44.840614 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:44.840552    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edf26460-8713-439a-bd02-f26f2f81f554-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft\" (UID: \"edf26460-8713-439a-bd02-f26f2f81f554\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"
Sep 11 11:53:44.922089 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:44.922055    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:44.941783 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:44.941760    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edf26460-8713-439a-bd02-f26f2f81f554-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft\" (UID: \"edf26460-8713-439a-bd02-f26f2f81f554\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"
Sep 11 11:53:44.941896 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:44.941802    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vpz8\" (UniqueName: \"kubernetes.io/projected/edf26460-8713-439a-bd02-f26f2f81f554-kube-api-access-6vpz8\") pod \"maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft\" (UID: \"edf26460-8713-439a-bd02-f26f2f81f554\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"
Sep 11 11:53:44.941896 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:44.941866    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edf26460-8713-439a-bd02-f26f2f81f554-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft\" (UID: \"edf26460-8713-439a-bd02-f26f2f81f554\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"
Sep 11 11:53:44.941896 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:44.941884    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edf26460-8713-439a-bd02-f26f2f81f554-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft\" (UID: \"edf26460-8713-439a-bd02-f26f2f81f554\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"
Sep 11 11:53:44.942033 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:53:44.942018    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-serving-cert: secret "maas-api-e2e-shared-b-w2-6daa53-serving-cert" not found
Sep 11 11:53:44.942094 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:53:44.942085    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edf26460-8713-439a-bd02-f26f2f81f554-maas-api-tls podName:edf26460-8713-439a-bd02-f26f2f81f554 nodeName:}" failed. No retries permitted until 2026-09-11 11:53:45.442067317 +0000 UTC m=+1165.862673336 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/edf26460-8713-439a-bd02-f26f2f81f554-maas-api-tls") pod "maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft" (UID: "edf26460-8713-439a-bd02-f26f2f81f554") : secret "maas-api-e2e-shared-b-w2-6daa53-serving-cert" not found
Sep 11 11:53:44.942513 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:44.942491    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edf26460-8713-439a-bd02-f26f2f81f554-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft\" (UID: \"edf26460-8713-439a-bd02-f26f2f81f554\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"
Sep 11 11:53:44.944310 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:44.944292    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edf26460-8713-439a-bd02-f26f2f81f554-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft\" (UID: \"edf26460-8713-439a-bd02-f26f2f81f554\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"
Sep 11 11:53:44.958138 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:44.958115    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vpz8\" (UniqueName: \"kubernetes.io/projected/edf26460-8713-439a-bd02-f26f2f81f554-kube-api-access-6vpz8\") pod \"maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft\" (UID: \"edf26460-8713-439a-bd02-f26f2f81f554\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"
Sep 11 11:53:45.194224 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:45.194141    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:45.369059 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:45.369021    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:45.447515 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:45.447439    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edf26460-8713-439a-bd02-f26f2f81f554-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft\" (UID: \"edf26460-8713-439a-bd02-f26f2f81f554\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"
Sep 11 11:53:45.449941 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:45.449922    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edf26460-8713-439a-bd02-f26f2f81f554-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft\" (UID: \"edf26460-8713-439a-bd02-f26f2f81f554\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"
Sep 11 11:53:45.488628 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:45.488599    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:45.661964 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:45.661936    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"
Sep 11 11:53:45.695853 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:45.695826    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl"]
Sep 11 11:53:45.699904 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:45.699840    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl"
Sep 11 11:53:45.702498 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:45.702478    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-6daa53-dockercfg-lhd4x\""
Sep 11 11:53:45.702623 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:45.702579    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-6daa53\""
Sep 11 11:53:45.716625 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:45.716562    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl"]
Sep 11 11:53:45.851599 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:45.851560    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czf6h\" (UniqueName: \"kubernetes.io/projected/0331ad11-878f-409a-8856-1c40b7d8c7a4-kube-api-access-czf6h\") pod \"payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl\" (UID: \"0331ad11-878f-409a-8856-1c40b7d8c7a4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl"
Sep 11 11:53:45.851988 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:45.851692    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0331ad11-878f-409a-8856-1c40b7d8c7a4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl\" (UID: \"0331ad11-878f-409a-8856-1c40b7d8c7a4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl"
Sep 11 11:53:45.944561 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:45.944535    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"]
Sep 11 11:53:45.946839 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:53:45.946812    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedf26460_8713_439a_bd02_f26f2f81f554.slice/crio-b0e7bd384f34191d5d17463cce52dc35358b2692d73f68db239b7e2da5f34a00 WatchSource:0}: Error finding container b0e7bd384f34191d5d17463cce52dc35358b2692d73f68db239b7e2da5f34a00: Status 404 returned error can't find the container with id b0e7bd384f34191d5d17463cce52dc35358b2692d73f68db239b7e2da5f34a00
Sep 11 11:53:45.952586 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:45.952561    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czf6h\" (UniqueName: \"kubernetes.io/projected/0331ad11-878f-409a-8856-1c40b7d8c7a4-kube-api-access-czf6h\") pod \"payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl\" (UID: \"0331ad11-878f-409a-8856-1c40b7d8c7a4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl"
Sep 11 11:53:45.952727 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:45.952708    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0331ad11-878f-409a-8856-1c40b7d8c7a4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl\" (UID: \"0331ad11-878f-409a-8856-1c40b7d8c7a4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl"
Sep 11 11:53:45.953327 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:45.953304    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0331ad11-878f-409a-8856-1c40b7d8c7a4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl\" (UID: \"0331ad11-878f-409a-8856-1c40b7d8c7a4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl"
Sep 11 11:53:45.966327 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:45.966308    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czf6h\" (UniqueName: \"kubernetes.io/projected/0331ad11-878f-409a-8856-1c40b7d8c7a4-kube-api-access-czf6h\") pod \"payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl\" (UID: \"0331ad11-878f-409a-8856-1c40b7d8c7a4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl"
Sep 11 11:53:45.972706 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:45.972683    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft" event={"ID":"edf26460-8713-439a-bd02-f26f2f81f554","Type":"ContainerStarted","Data":"b0e7bd384f34191d5d17463cce52dc35358b2692d73f68db239b7e2da5f34a00"}
Sep 11 11:53:46.013833 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:46.013810    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl"
Sep 11 11:53:46.293973 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:46.293948    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl"]
Sep 11 11:53:46.295261 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:53:46.295234    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0331ad11_878f_409a_8856_1c40b7d8c7a4.slice/crio-2877c131aa2bb7883dddff8c10c8dfc269a80c3018f1d1188f29fcf652fe02e2 WatchSource:0}: Error finding container 2877c131aa2bb7883dddff8c10c8dfc269a80c3018f1d1188f29fcf652fe02e2: Status 404 returned error can't find the container with id 2877c131aa2bb7883dddff8c10c8dfc269a80c3018f1d1188f29fcf652fe02e2
Sep 11 11:53:46.980861 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:46.980823    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl" event={"ID":"0331ad11-878f-409a-8856-1c40b7d8c7a4","Type":"ContainerStarted","Data":"568bb88e77bde93dafb385a30b7d641b548ba48f759632a117709ef6d8e994f4"}
Sep 11 11:53:46.981267 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:46.980892    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl" event={"ID":"0331ad11-878f-409a-8856-1c40b7d8c7a4","Type":"ContainerStarted","Data":"2877c131aa2bb7883dddff8c10c8dfc269a80c3018f1d1188f29fcf652fe02e2"}
Sep 11 11:53:46.981267 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:46.980926    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl"
Sep 11 11:53:46.982424 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:46.982401    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft" event={"ID":"edf26460-8713-439a-bd02-f26f2f81f554","Type":"ContainerStarted","Data":"445e820b2a478646e289dc07ece90794bc0ace8d5097a9eed86dbddd2b4b8bb9"}
Sep 11 11:53:46.982551 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:46.982520    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"
Sep 11 11:53:47.005513 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:47.005462    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl" podStartSLOduration=2.005443513 podStartE2EDuration="2.005443513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:53:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:47.00312922 +0000 UTC m=+1167.423735248" watchObservedRunningTime="2026-09-11 11:53:47.005443513 +0000 UTC m=+1167.426049541"
Sep 11 11:53:52.993336 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:52.993308    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"
Sep 11 11:53:53.014335 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:53.014293    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft" podStartSLOduration=8.711491473 podStartE2EDuration="9.014277209s" totalImagesPullingTime="302.785736ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:53:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:53:47.036131015 +0000 UTC m=+1167.456737043" watchObservedRunningTime="2026-09-11 11:53:53.014277209 +0000 UTC m=+1173.434883237"
Sep 11 11:53:55.058865 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:55.058828    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:53:57.990198 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:53:57.990170    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl"
Sep 11 11:54:00.907185 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:54:00.907148    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:05.464107 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:54:05.464062    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:07.426801 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:54:07.426765    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:13.453397 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:54:13.453355    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:21.659739 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:54:21.659703    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:22.920331 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:54:22.920297    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:23.255712 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:54:23.255681    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:23.417700 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:54:23.417666    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:26.767559 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:54:26.767519    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:26.940291 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:54:26.940258    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:27.979037 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:54:27.979000    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:28.077680 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:54:28.077612    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:28.176962 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:54:28.176933    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:28.657551 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:54:28.657503    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:28.755096 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:54:28.755061    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:39.952371 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:54:39.952318    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:49.077732 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:54:49.077692    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:49.766948 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:54:49.766919    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:54:51.688930 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:54:51.688893    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:05.795024 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:05.794997    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"]
Sep 11 11:55:05.798781 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:05.798764    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:05.807054 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:05.807018    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-307b7f39\"/\"default-dockercfg-fg8b7\""
Sep 11 11:55:05.807173 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:05.807062    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-307b7f39\"/\"kube-root-ca.crt\""
Sep 11 11:55:05.807173 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:05.807026    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-307b7f39\"/\"openshift-service-ca.crt\""
Sep 11 11:55:05.808010 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:05.807994    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-307b7f39\"/\"e2e-lifecycle-model-307b7f39-kserve-self-signed-certs\""
Sep 11 11:55:05.819508 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:05.819485    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"]
Sep 11 11:55:05.905849 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:05.905819    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-model-cache\") pod \"e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") " pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:05.906012 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:05.905863    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36127080-1a7c-40d4-91af-ac5d33e99a05-tls-certs\") pod \"e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") " pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:05.906012 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:05.905979    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s52b\" (UniqueName: \"kubernetes.io/projected/36127080-1a7c-40d4-91af-ac5d33e99a05-kube-api-access-2s52b\") pod \"e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") " pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:05.906093 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:05.906016    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-tmp-dir\") pod \"e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") " pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:05.906093 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:05.906080    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-dshm\") pod \"e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") " pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:05.906159 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:05.906105    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-home\") pod \"e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") " pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:05.936917 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:05.936892    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:06.007183 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:06.007156    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-model-cache\") pod \"e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") " pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:06.007331 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:06.007205    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36127080-1a7c-40d4-91af-ac5d33e99a05-tls-certs\") pod \"e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") " pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:06.007331 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:06.007266    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s52b\" (UniqueName: \"kubernetes.io/projected/36127080-1a7c-40d4-91af-ac5d33e99a05-kube-api-access-2s52b\") pod \"e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") " pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:06.007331 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:06.007299    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-tmp-dir\") pod \"e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") " pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:06.007495 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:06.007357    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-dshm\") pod \"e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") " pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:06.007495 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:06.007395    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-home\") pod \"e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") " pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:06.007662 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:06.007606    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-model-cache\") pod \"e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") " pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:06.007775 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:06.007752    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-tmp-dir\") pod \"e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") " pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:06.007840 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:06.007794    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-home\") pod \"e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") " pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:06.009927 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:06.009901    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-dshm\") pod \"e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") " pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:06.010035 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:06.010020    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36127080-1a7c-40d4-91af-ac5d33e99a05-tls-certs\") pod \"e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") " pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:06.015825 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:06.015805    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s52b\" (UniqueName: \"kubernetes.io/projected/36127080-1a7c-40d4-91af-ac5d33e99a05-kube-api-access-2s52b\") pod \"e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") " pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:06.108304 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:06.108213    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:06.382491 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:06.382469    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"]
Sep 11 11:55:06.384174 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:55:06.384146    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36127080_1a7c_40d4_91af_ac5d33e99a05.slice/crio-233c117ad87643b18131545ccd424da581166ac076ae237c5d50dd9010526914 WatchSource:0}: Error finding container 233c117ad87643b18131545ccd424da581166ac076ae237c5d50dd9010526914: Status 404 returned error can't find the container with id 233c117ad87643b18131545ccd424da581166ac076ae237c5d50dd9010526914
Sep 11 11:55:07.397617 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:07.397577    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm" event={"ID":"36127080-1a7c-40d4-91af-ac5d33e99a05","Type":"ContainerStarted","Data":"dc943b486a5d3d561ab77b057c2f18a6d71607796aecc5c9a2ffb3aa135cf3f7"}
Sep 11 11:55:07.398017 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:07.397652    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm" event={"ID":"36127080-1a7c-40d4-91af-ac5d33e99a05","Type":"ContainerStarted","Data":"233c117ad87643b18131545ccd424da581166ac076ae237c5d50dd9010526914"}
Sep 11 11:55:07.418583 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:07.418537    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm" podStartSLOduration=2.418523142 podStartE2EDuration="2.418523142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:55:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:55:07.416295073 +0000 UTC m=+1247.836901113" watchObservedRunningTime="2026-09-11 11:55:07.418523142 +0000 UTC m=+1247.839129170"
Sep 11 11:55:14.426764 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.426684    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"]
Sep 11 11:55:14.430906 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.430881    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:55:14.434286 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.434261    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-117711\"/\"openshift-service-ca.crt\""
Sep 11 11:55:14.434415 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.434383    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-117711\"/\"auth-test-model-117711-kserve-self-signed-certs\""
Sep 11 11:55:14.434495 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.434480    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-117711\"/\"kube-root-ca.crt\""
Sep 11 11:55:14.434586 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.434571    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-117711\"/\"default-dockercfg-ndglz\""
Sep 11 11:55:14.440532 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.440507    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"]
Sep 11 11:55:14.537148 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.537114    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:55:14.594090 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.594055    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-model-cache\") pod \"auth-test-model-117711-kserve-5cb75bf95f-6ht8f\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") " pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:55:14.594090 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.594088    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-dshm\") pod \"auth-test-model-117711-kserve-5cb75bf95f-6ht8f\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") " pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:55:14.594308 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.594203    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-home\") pod \"auth-test-model-117711-kserve-5cb75bf95f-6ht8f\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") " pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:55:14.594308 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.594242    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vv4r\" (UniqueName: \"kubernetes.io/projected/ca02657f-43d1-4a8c-a19b-3bb817b227ef-kube-api-access-7vv4r\") pod \"auth-test-model-117711-kserve-5cb75bf95f-6ht8f\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") " pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:55:14.594308 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.594294    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca02657f-43d1-4a8c-a19b-3bb817b227ef-tls-certs\") pod \"auth-test-model-117711-kserve-5cb75bf95f-6ht8f\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") " pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:55:14.594437 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.594360    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-tmp-dir\") pod \"auth-test-model-117711-kserve-5cb75bf95f-6ht8f\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") " pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:55:14.695890 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.695813    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-dshm\") pod \"auth-test-model-117711-kserve-5cb75bf95f-6ht8f\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") " pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:55:14.695890 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.695866    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-home\") pod \"auth-test-model-117711-kserve-5cb75bf95f-6ht8f\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") " pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:55:14.695890 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.695886    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vv4r\" (UniqueName: \"kubernetes.io/projected/ca02657f-43d1-4a8c-a19b-3bb817b227ef-kube-api-access-7vv4r\") pod \"auth-test-model-117711-kserve-5cb75bf95f-6ht8f\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") " pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:55:14.696161 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.695909    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca02657f-43d1-4a8c-a19b-3bb817b227ef-tls-certs\") pod \"auth-test-model-117711-kserve-5cb75bf95f-6ht8f\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") " pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:55:14.696161 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.695939    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-tmp-dir\") pod \"auth-test-model-117711-kserve-5cb75bf95f-6ht8f\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") " pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:55:14.696161 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.696037    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-model-cache\") pod \"auth-test-model-117711-kserve-5cb75bf95f-6ht8f\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") " pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:55:14.696327 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.696298    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-home\") pod \"auth-test-model-117711-kserve-5cb75bf95f-6ht8f\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") " pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:55:14.696393 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.696331    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-model-cache\") pod \"auth-test-model-117711-kserve-5cb75bf95f-6ht8f\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") " pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:55:14.696459 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.696431    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-tmp-dir\") pod \"auth-test-model-117711-kserve-5cb75bf95f-6ht8f\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") " pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:55:14.698193 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.698169    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-dshm\") pod \"auth-test-model-117711-kserve-5cb75bf95f-6ht8f\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") " pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:55:14.698504 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.698485    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca02657f-43d1-4a8c-a19b-3bb817b227ef-tls-certs\") pod \"auth-test-model-117711-kserve-5cb75bf95f-6ht8f\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") " pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:55:14.704555 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.704525    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vv4r\" (UniqueName: \"kubernetes.io/projected/ca02657f-43d1-4a8c-a19b-3bb817b227ef-kube-api-access-7vv4r\") pod \"auth-test-model-117711-kserve-5cb75bf95f-6ht8f\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") " pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:55:14.742506 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:14.742482    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:55:15.020226 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:15.020199    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"]
Sep 11 11:55:15.022825 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:55:15.022798    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca02657f_43d1_4a8c_a19b_3bb817b227ef.slice/crio-70863942a1b05aeb5cc7e1877347495d4201348430ab36c59476435f5a7a0621 WatchSource:0}: Error finding container 70863942a1b05aeb5cc7e1877347495d4201348430ab36c59476435f5a7a0621: Status 404 returned error can't find the container with id 70863942a1b05aeb5cc7e1877347495d4201348430ab36c59476435f5a7a0621
Sep 11 11:55:15.440432 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:15.440338    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f" event={"ID":"ca02657f-43d1-4a8c-a19b-3bb817b227ef","Type":"ContainerStarted","Data":"59f20f8386624886fa89f70ff94699b8ab3a9230f4fd8f79e2f5d43a6f22be2a"}
Sep 11 11:55:15.440432 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:15.440384    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f" event={"ID":"ca02657f-43d1-4a8c-a19b-3bb817b227ef","Type":"ContainerStarted","Data":"70863942a1b05aeb5cc7e1877347495d4201348430ab36c59476435f5a7a0621"}
Sep 11 11:55:15.458261 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:15.458213    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f" podStartSLOduration=1.458198997 podStartE2EDuration="1.458198997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:55:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:55:15.45655918 +0000 UTC m=+1255.877165207" watchObservedRunningTime="2026-09-11 11:55:15.458198997 +0000 UTC m=+1255.878805023"
Sep 11 11:55:16.108784 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:16.108754    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:16.108992 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:16.108803    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:16.121682 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:16.121652    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:16.457061 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:16.456977    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:55:24.743556 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:24.743522    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:55:24.744004 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:24.743579    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:55:24.756531 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:24.756507    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:55:25.508234 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:55:25.508208    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:56:01.965909 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:01.965868    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:02.080906 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:02.080863    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:02.341500 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:02.341464    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:02.535044 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:02.535004    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:05.562021 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:05.561990    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:10.524999 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:10.524962    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"]
Sep 11 11:56:10.655989 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:10.655955    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:21.919982 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:21.919946    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:21.978685 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:21.978655    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:25.544811 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.544771    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm" podUID="36127080-1a7c-40d4-91af-ac5d33e99a05" containerName="main" containerID="cri-o://dc943b486a5d3d561ab77b057c2f18a6d71607796aecc5c9a2ffb3aa135cf3f7" gracePeriod=45
Sep 11 11:56:25.802506 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.802446    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:56:25.811577 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.811553    2575 generic.go:356] "Generic (PLEG): container finished" podID="36127080-1a7c-40d4-91af-ac5d33e99a05" containerID="dc943b486a5d3d561ab77b057c2f18a6d71607796aecc5c9a2ffb3aa135cf3f7" exitCode=0
Sep 11 11:56:25.811715 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.811590    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"
Sep 11 11:56:25.811715 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.811626    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm" event={"ID":"36127080-1a7c-40d4-91af-ac5d33e99a05","Type":"ContainerDied","Data":"dc943b486a5d3d561ab77b057c2f18a6d71607796aecc5c9a2ffb3aa135cf3f7"}
Sep 11 11:56:25.811715 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.811678    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm" event={"ID":"36127080-1a7c-40d4-91af-ac5d33e99a05","Type":"ContainerDied","Data":"233c117ad87643b18131545ccd424da581166ac076ae237c5d50dd9010526914"}
Sep 11 11:56:25.811715 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.811696    2575 scope.go:117] "RemoveContainer" containerID="dc943b486a5d3d561ab77b057c2f18a6d71607796aecc5c9a2ffb3aa135cf3f7"
Sep 11 11:56:25.821157 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.821119    2575 scope.go:117] "RemoveContainer" containerID="dc943b486a5d3d561ab77b057c2f18a6d71607796aecc5c9a2ffb3aa135cf3f7"
Sep 11 11:56:25.821484 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:56:25.821463    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc943b486a5d3d561ab77b057c2f18a6d71607796aecc5c9a2ffb3aa135cf3f7\": container with ID starting with dc943b486a5d3d561ab77b057c2f18a6d71607796aecc5c9a2ffb3aa135cf3f7 not found: ID does not exist" containerID="dc943b486a5d3d561ab77b057c2f18a6d71607796aecc5c9a2ffb3aa135cf3f7"
Sep 11 11:56:25.821568 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.821493    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc943b486a5d3d561ab77b057c2f18a6d71607796aecc5c9a2ffb3aa135cf3f7"} err="failed to get container status \"dc943b486a5d3d561ab77b057c2f18a6d71607796aecc5c9a2ffb3aa135cf3f7\": rpc error: code = NotFound desc = could not find container \"dc943b486a5d3d561ab77b057c2f18a6d71607796aecc5c9a2ffb3aa135cf3f7\": container with ID starting with dc943b486a5d3d561ab77b057c2f18a6d71607796aecc5c9a2ffb3aa135cf3f7 not found: ID does not exist"
Sep 11 11:56:25.880648 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.880602    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-dshm\") pod \"36127080-1a7c-40d4-91af-ac5d33e99a05\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") "
Sep 11 11:56:25.880818 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.880671    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-model-cache\") pod \"36127080-1a7c-40d4-91af-ac5d33e99a05\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") "
Sep 11 11:56:25.880818 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.880712    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-tmp-dir\") pod \"36127080-1a7c-40d4-91af-ac5d33e99a05\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") "
Sep 11 11:56:25.880818 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.880746    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-home\") pod \"36127080-1a7c-40d4-91af-ac5d33e99a05\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") "
Sep 11 11:56:25.880818 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.880786    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2s52b\" (UniqueName: \"kubernetes.io/projected/36127080-1a7c-40d4-91af-ac5d33e99a05-kube-api-access-2s52b\") pod \"36127080-1a7c-40d4-91af-ac5d33e99a05\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") "
Sep 11 11:56:25.881049 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.880874    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36127080-1a7c-40d4-91af-ac5d33e99a05-tls-certs\") pod \"36127080-1a7c-40d4-91af-ac5d33e99a05\" (UID: \"36127080-1a7c-40d4-91af-ac5d33e99a05\") "
Sep 11 11:56:25.881049 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.880999    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-model-cache" (OuterVolumeSpecName: "model-cache") pod "36127080-1a7c-40d4-91af-ac5d33e99a05" (UID: "36127080-1a7c-40d4-91af-ac5d33e99a05"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:56:25.881049 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.881018    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "36127080-1a7c-40d4-91af-ac5d33e99a05" (UID: "36127080-1a7c-40d4-91af-ac5d33e99a05"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:56:25.881253 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.881235    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-model-cache\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:56:25.881306 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.881260    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-tmp-dir\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:56:25.881306 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.881256    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-home" (OuterVolumeSpecName: "home") pod "36127080-1a7c-40d4-91af-ac5d33e99a05" (UID: "36127080-1a7c-40d4-91af-ac5d33e99a05"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:56:25.883257 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.883236    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36127080-1a7c-40d4-91af-ac5d33e99a05-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "36127080-1a7c-40d4-91af-ac5d33e99a05" (UID: "36127080-1a7c-40d4-91af-ac5d33e99a05"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:56:25.883387 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.883369    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-dshm" (OuterVolumeSpecName: "dshm") pod "36127080-1a7c-40d4-91af-ac5d33e99a05" (UID: "36127080-1a7c-40d4-91af-ac5d33e99a05"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:56:25.883387 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.883374    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36127080-1a7c-40d4-91af-ac5d33e99a05-kube-api-access-2s52b" (OuterVolumeSpecName: "kube-api-access-2s52b") pod "36127080-1a7c-40d4-91af-ac5d33e99a05" (UID: "36127080-1a7c-40d4-91af-ac5d33e99a05"). InnerVolumeSpecName "kube-api-access-2s52b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:56:25.982488 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.982460    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-home\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:56:25.982488 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.982486    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2s52b\" (UniqueName: \"kubernetes.io/projected/36127080-1a7c-40d4-91af-ac5d33e99a05-kube-api-access-2s52b\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:56:25.982690 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.982498    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36127080-1a7c-40d4-91af-ac5d33e99a05-tls-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:56:25.982690 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:25.982506    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36127080-1a7c-40d4-91af-ac5d33e99a05-dshm\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:56:26.128896 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:26.128866    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"]
Sep 11 11:56:26.133164 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:26.133140    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-307b7f39/e2e-lifecycle-model-307b7f39-kserve-6c79776655-xpjpm"]
Sep 11 11:56:28.082492 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:28.082453    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36127080-1a7c-40d4-91af-ac5d33e99a05" path="/var/lib/kubelet/pods/36127080-1a7c-40d4-91af-ac5d33e99a05/volumes"
Sep 11 11:56:46.900143 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:46.900067    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:47.999945 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:47.999914    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:53.599822 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:53.599777    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:53.718816 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:53.718781    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:53.916748 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:53.916657    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:55.087048 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:55.087009    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:55.185706 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:55.185674    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:55.285621 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:55.285578    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:55.846283 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:55.846258    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:55.946030 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:55.945998    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:56.053449 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:56.053425    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:57.210782 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:57.210738    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:59.843926 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:59.843889    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:56:59.945357 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:56:59.945326    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:03.204847 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:03.204810    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:03.269459 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:03.269426    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:03.871260 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:03.871228    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:09.592571 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:09.592538    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:09.658206 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:09.658171    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:09.750357 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:09.750330    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:09.879180 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:09.879096    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:20.263447 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:20.263412    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:20.467546 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:20.467486    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:28.852139 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:28.852100    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:29.734020 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:29.733980    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"]
Sep 11 11:57:29.754250 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:29.754220    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:41.387295 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:41.387246    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:42.252066 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:42.252031    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:44.754557 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:44.754498    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f" podUID="ca02657f-43d1-4a8c-a19b-3bb817b227ef" containerName="main" containerID="cri-o://59f20f8386624886fa89f70ff94699b8ab3a9230f4fd8f79e2f5d43a6f22be2a" gracePeriod=45
Sep 11 11:57:44.999062 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:44.999039    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:57:45.156600 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.156499    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca02657f-43d1-4a8c-a19b-3bb817b227ef-tls-certs\") pod \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") "
Sep 11 11:57:45.156600 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.156567    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-dshm\") pod \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") "
Sep 11 11:57:45.156866 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.156718    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-tmp-dir\") pod \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") "
Sep 11 11:57:45.156866 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.156750    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-model-cache\") pod \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") "
Sep 11 11:57:45.156866 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.156777    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-home\") pod \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") "
Sep 11 11:57:45.156866 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.156806    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vv4r\" (UniqueName: \"kubernetes.io/projected/ca02657f-43d1-4a8c-a19b-3bb817b227ef-kube-api-access-7vv4r\") pod \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\" (UID: \"ca02657f-43d1-4a8c-a19b-3bb817b227ef\") "
Sep 11 11:57:45.157066 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.157007    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ca02657f-43d1-4a8c-a19b-3bb817b227ef" (UID: "ca02657f-43d1-4a8c-a19b-3bb817b227ef"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:57:45.157066 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.157037    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-model-cache" (OuterVolumeSpecName: "model-cache") pod "ca02657f-43d1-4a8c-a19b-3bb817b227ef" (UID: "ca02657f-43d1-4a8c-a19b-3bb817b227ef"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:57:45.157066 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.157046    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-home" (OuterVolumeSpecName: "home") pod "ca02657f-43d1-4a8c-a19b-3bb817b227ef" (UID: "ca02657f-43d1-4a8c-a19b-3bb817b227ef"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:57:45.157185 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.157163    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-model-cache\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:57:45.157185 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.157176    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-home\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:57:45.157185 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.157184    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-tmp-dir\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:57:45.158968 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.158940    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca02657f-43d1-4a8c-a19b-3bb817b227ef-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ca02657f-43d1-4a8c-a19b-3bb817b227ef" (UID: "ca02657f-43d1-4a8c-a19b-3bb817b227ef"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:57:45.159098 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.159053    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-dshm" (OuterVolumeSpecName: "dshm") pod "ca02657f-43d1-4a8c-a19b-3bb817b227ef" (UID: "ca02657f-43d1-4a8c-a19b-3bb817b227ef"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 11:57:45.159098 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.159054    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca02657f-43d1-4a8c-a19b-3bb817b227ef-kube-api-access-7vv4r" (OuterVolumeSpecName: "kube-api-access-7vv4r") pod "ca02657f-43d1-4a8c-a19b-3bb817b227ef" (UID: "ca02657f-43d1-4a8c-a19b-3bb817b227ef"). InnerVolumeSpecName "kube-api-access-7vv4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:57:45.232388 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.232353    2575 generic.go:356] "Generic (PLEG): container finished" podID="ca02657f-43d1-4a8c-a19b-3bb817b227ef" containerID="59f20f8386624886fa89f70ff94699b8ab3a9230f4fd8f79e2f5d43a6f22be2a" exitCode=0
Sep 11 11:57:45.232557 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.232399    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"
Sep 11 11:57:45.232557 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.232447    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f" event={"ID":"ca02657f-43d1-4a8c-a19b-3bb817b227ef","Type":"ContainerDied","Data":"59f20f8386624886fa89f70ff94699b8ab3a9230f4fd8f79e2f5d43a6f22be2a"}
Sep 11 11:57:45.232557 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.232490    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f" event={"ID":"ca02657f-43d1-4a8c-a19b-3bb817b227ef","Type":"ContainerDied","Data":"70863942a1b05aeb5cc7e1877347495d4201348430ab36c59476435f5a7a0621"}
Sep 11 11:57:45.232557 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.232505    2575 scope.go:117] "RemoveContainer" containerID="59f20f8386624886fa89f70ff94699b8ab3a9230f4fd8f79e2f5d43a6f22be2a"
Sep 11 11:57:45.245848 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.245829    2575 scope.go:117] "RemoveContainer" containerID="59f20f8386624886fa89f70ff94699b8ab3a9230f4fd8f79e2f5d43a6f22be2a"
Sep 11 11:57:45.246139 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:57:45.246120    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59f20f8386624886fa89f70ff94699b8ab3a9230f4fd8f79e2f5d43a6f22be2a\": container with ID starting with 59f20f8386624886fa89f70ff94699b8ab3a9230f4fd8f79e2f5d43a6f22be2a not found: ID does not exist" containerID="59f20f8386624886fa89f70ff94699b8ab3a9230f4fd8f79e2f5d43a6f22be2a"
Sep 11 11:57:45.246212 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.246150    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59f20f8386624886fa89f70ff94699b8ab3a9230f4fd8f79e2f5d43a6f22be2a"} err="failed to get container status \"59f20f8386624886fa89f70ff94699b8ab3a9230f4fd8f79e2f5d43a6f22be2a\": rpc error: code = NotFound desc = could not find container \"59f20f8386624886fa89f70ff94699b8ab3a9230f4fd8f79e2f5d43a6f22be2a\": container with ID starting with 59f20f8386624886fa89f70ff94699b8ab3a9230f4fd8f79e2f5d43a6f22be2a not found: ID does not exist"
Sep 11 11:57:45.254944 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.254922    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"]
Sep 11 11:57:45.256922 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.256899    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-117711/auth-test-model-117711-kserve-5cb75bf95f-6ht8f"]
Sep 11 11:57:45.258121 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.258105    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vv4r\" (UniqueName: \"kubernetes.io/projected/ca02657f-43d1-4a8c-a19b-3bb817b227ef-kube-api-access-7vv4r\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:57:45.258173 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.258124    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca02657f-43d1-4a8c-a19b-3bb817b227ef-tls-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:57:45.258173 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:45.258136    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca02657f-43d1-4a8c-a19b-3bb817b227ef-dshm\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:57:46.082220 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:46.082189    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca02657f-43d1-4a8c-a19b-3bb817b227ef" path="/var/lib/kubelet/pods/ca02657f-43d1-4a8c-a19b-3bb817b227ef/volumes"
Sep 11 11:57:47.226230 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:47.226198    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:47.328349 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:47.328318    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:47.556553 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:47.556514    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:47.857076 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:47.856977    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:48.626177 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:48.626147    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:51.343706 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:51.343663    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:51.444248 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:51.444196    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:52.596989 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:52.596949    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:52.692298 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:52.692255    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:53.080787 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:53.080754    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:53.180344 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:53.180308    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:57:53.219291 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:53.219262    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4"]
Sep 11 11:57:53.219719 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:53.219704    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36127080-1a7c-40d4-91af-ac5d33e99a05" containerName="main"
Sep 11 11:57:53.219719 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:53.219719    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="36127080-1a7c-40d4-91af-ac5d33e99a05" containerName="main"
Sep 11 11:57:53.219817 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:53.219746    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca02657f-43d1-4a8c-a19b-3bb817b227ef" containerName="main"
Sep 11 11:57:53.219817 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:53.219759    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca02657f-43d1-4a8c-a19b-3bb817b227ef" containerName="main"
Sep 11 11:57:53.219890 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:53.219820    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="36127080-1a7c-40d4-91af-ac5d33e99a05" containerName="main"
Sep 11 11:57:53.219890 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:53.219830    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca02657f-43d1-4a8c-a19b-3bb817b227ef" containerName="main"
Sep 11 11:57:53.222993 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:53.222975    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4"
Sep 11 11:57:53.225402 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:53.225386    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-f9def1-dockercfg-g47cc\""
Sep 11 11:57:53.225840 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:53.225826    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-f9def1\""
Sep 11 11:57:53.238471 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:53.238450    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4"]
Sep 11 11:57:53.335183 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:53.335090    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj5tq\" (UniqueName: \"kubernetes.io/projected/e535cd14-2818-486e-b28e-2cfca1ac882a-kube-api-access-fj5tq\") pod \"payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4\" (UID: \"e535cd14-2818-486e-b28e-2cfca1ac882a\") " pod="openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4"
Sep 11 11:57:53.335183 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:53.335168    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e535cd14-2818-486e-b28e-2cfca1ac882a-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4\" (UID: \"e535cd14-2818-486e-b28e-2cfca1ac882a\") " pod="openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4"
Sep 11 11:57:53.436560 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:53.436521    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj5tq\" (UniqueName: \"kubernetes.io/projected/e535cd14-2818-486e-b28e-2cfca1ac882a-kube-api-access-fj5tq\") pod \"payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4\" (UID: \"e535cd14-2818-486e-b28e-2cfca1ac882a\") " pod="openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4"
Sep 11 11:57:53.436560 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:53.436561    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e535cd14-2818-486e-b28e-2cfca1ac882a-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4\" (UID: \"e535cd14-2818-486e-b28e-2cfca1ac882a\") " pod="openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4"
Sep 11 11:57:53.437316 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:53.437297    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e535cd14-2818-486e-b28e-2cfca1ac882a-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4\" (UID: \"e535cd14-2818-486e-b28e-2cfca1ac882a\") " pod="openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4"
Sep 11 11:57:53.444844 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:53.444824    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj5tq\" (UniqueName: \"kubernetes.io/projected/e535cd14-2818-486e-b28e-2cfca1ac882a-kube-api-access-fj5tq\") pod \"payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4\" (UID: \"e535cd14-2818-486e-b28e-2cfca1ac882a\") " pod="openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4"
Sep 11 11:57:53.533003 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:53.532968    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4"
Sep 11 11:57:53.806494 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:53.806465    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4"]
Sep 11 11:57:53.808461 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:57:53.808437    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode535cd14_2818_486e_b28e_2cfca1ac882a.slice/crio-50c9ec5530c2fb88d76fe915b910be9fc6aa6a821950cb3e573f846b16cd6790 WatchSource:0}: Error finding container 50c9ec5530c2fb88d76fe915b910be9fc6aa6a821950cb3e573f846b16cd6790: Status 404 returned error can't find the container with id 50c9ec5530c2fb88d76fe915b910be9fc6aa6a821950cb3e573f846b16cd6790
Sep 11 11:57:54.283976 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:54.283945    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4" event={"ID":"e535cd14-2818-486e-b28e-2cfca1ac882a","Type":"ContainerStarted","Data":"65650f7ed0127e2bdef4895a7f59ba160dd7801856081be28d8cfb88e9eddf25"}
Sep 11 11:57:54.284177 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:54.284016    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4" event={"ID":"e535cd14-2818-486e-b28e-2cfca1ac882a","Type":"ContainerStarted","Data":"50c9ec5530c2fb88d76fe915b910be9fc6aa6a821950cb3e573f846b16cd6790"}
Sep 11 11:57:54.284177 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:54.284067    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4"
Sep 11 11:57:54.302399 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:57:54.302352    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4" podStartSLOduration=1.302337364 podStartE2EDuration="1.302337364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:57:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:57:54.301113997 +0000 UTC m=+1414.721720059" watchObservedRunningTime="2026-09-11 11:57:54.302337364 +0000 UTC m=+1414.722943391"
Sep 11 11:58:00.158016 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:00.157983    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:05.292123 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:05.292093    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4"
Sep 11 11:58:08.307326 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:08.307292    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:08.410469 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:08.410435    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:16.251972 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:16.251895    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:16.725851 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:16.725745    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:17.025796 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:17.025769    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:17.389101 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:17.389017    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:20.548206 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:20.548125    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:20.644568 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:20.644537    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:21.695493 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:21.695458    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:21.797254 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:21.797222    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:21.939378 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:21.939347    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:22.198521 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.198486    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:22.299001 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.298967    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:22.321700 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.321655    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf"]
Sep 11 11:58:22.326275 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.326261    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf"
Sep 11 11:58:22.329270 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.329242    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-641847\""
Sep 11 11:58:22.329457 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.329443    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-641847-dockercfg-fz48f\""
Sep 11 11:58:22.334895 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.334858    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf"]
Sep 11 11:58:22.341171 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.341148    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75"]
Sep 11 11:58:22.344789 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.344772    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75"
Sep 11 11:58:22.359132 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.359096    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75"]
Sep 11 11:58:22.406682 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.406626    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/281f2d78-83e8-47ab-8f2c-0ca48db6353e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75\" (UID: \"281f2d78-83e8-47ab-8f2c-0ca48db6353e\") " pod="openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75"
Sep 11 11:58:22.406867 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.406826    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq6n7\" (UniqueName: \"kubernetes.io/projected/281f2d78-83e8-47ab-8f2c-0ca48db6353e-kube-api-access-sq6n7\") pod \"payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75\" (UID: \"281f2d78-83e8-47ab-8f2c-0ca48db6353e\") " pod="openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75"
Sep 11 11:58:22.406951 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.406928    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdr8t\" (UniqueName: \"kubernetes.io/projected/cc2df851-cac0-4266-9f57-32e4fc11db0e-kube-api-access-tdr8t\") pod \"payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf\" (UID: \"cc2df851-cac0-4266-9f57-32e4fc11db0e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf"
Sep 11 11:58:22.407001 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.406963    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc2df851-cac0-4266-9f57-32e4fc11db0e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf\" (UID: \"cc2df851-cac0-4266-9f57-32e4fc11db0e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf"
Sep 11 11:58:22.508342 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.508301    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdr8t\" (UniqueName: \"kubernetes.io/projected/cc2df851-cac0-4266-9f57-32e4fc11db0e-kube-api-access-tdr8t\") pod \"payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf\" (UID: \"cc2df851-cac0-4266-9f57-32e4fc11db0e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf"
Sep 11 11:58:22.508342 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.508341    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc2df851-cac0-4266-9f57-32e4fc11db0e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf\" (UID: \"cc2df851-cac0-4266-9f57-32e4fc11db0e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf"
Sep 11 11:58:22.508608 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.508361    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/281f2d78-83e8-47ab-8f2c-0ca48db6353e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75\" (UID: \"281f2d78-83e8-47ab-8f2c-0ca48db6353e\") " pod="openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75"
Sep 11 11:58:22.508608 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.508445    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq6n7\" (UniqueName: \"kubernetes.io/projected/281f2d78-83e8-47ab-8f2c-0ca48db6353e-kube-api-access-sq6n7\") pod \"payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75\" (UID: \"281f2d78-83e8-47ab-8f2c-0ca48db6353e\") " pod="openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75"
Sep 11 11:58:22.509179 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.509160    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/281f2d78-83e8-47ab-8f2c-0ca48db6353e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75\" (UID: \"281f2d78-83e8-47ab-8f2c-0ca48db6353e\") " pod="openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75"
Sep 11 11:58:22.509312 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.509287    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc2df851-cac0-4266-9f57-32e4fc11db0e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf\" (UID: \"cc2df851-cac0-4266-9f57-32e4fc11db0e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf"
Sep 11 11:58:22.518176 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.518146    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdr8t\" (UniqueName: \"kubernetes.io/projected/cc2df851-cac0-4266-9f57-32e4fc11db0e-kube-api-access-tdr8t\") pod \"payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf\" (UID: \"cc2df851-cac0-4266-9f57-32e4fc11db0e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf"
Sep 11 11:58:22.518176 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.518160    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq6n7\" (UniqueName: \"kubernetes.io/projected/281f2d78-83e8-47ab-8f2c-0ca48db6353e-kube-api-access-sq6n7\") pod \"payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75\" (UID: \"281f2d78-83e8-47ab-8f2c-0ca48db6353e\") " pod="openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75"
Sep 11 11:58:22.638292 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.638257    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf"
Sep 11 11:58:22.656751 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.656715    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75"
Sep 11 11:58:22.916866 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.916841    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf"]
Sep 11 11:58:22.918791 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:58:22.918764    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc2df851_cac0_4266_9f57_32e4fc11db0e.slice/crio-265a8ffae65f6a9674b1bb81f93f481d6bb2e80ef15626bf5520b4c1ff868f6d WatchSource:0}: Error finding container 265a8ffae65f6a9674b1bb81f93f481d6bb2e80ef15626bf5520b4c1ff868f6d: Status 404 returned error can't find the container with id 265a8ffae65f6a9674b1bb81f93f481d6bb2e80ef15626bf5520b4c1ff868f6d
Sep 11 11:58:22.941337 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:22.941312    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75"]
Sep 11 11:58:22.942726 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:58:22.942701    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod281f2d78_83e8_47ab_8f2c_0ca48db6353e.slice/crio-a5620bbbf0ec83533d993e659f8abe9e704877fe9a7020fedcb7ae7fbef4c27e WatchSource:0}: Error finding container a5620bbbf0ec83533d993e659f8abe9e704877fe9a7020fedcb7ae7fbef4c27e: Status 404 returned error can't find the container with id a5620bbbf0ec83533d993e659f8abe9e704877fe9a7020fedcb7ae7fbef4c27e
Sep 11 11:58:23.449980 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:23.449944    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf" event={"ID":"cc2df851-cac0-4266-9f57-32e4fc11db0e","Type":"ContainerStarted","Data":"6018dbe483a41dd6b41f3dff552f5b69742b05b44f0f09b00116a74a9a6c6bc5"}
Sep 11 11:58:23.450183 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:23.450007    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf"
Sep 11 11:58:23.450183 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:23.450034    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf" event={"ID":"cc2df851-cac0-4266-9f57-32e4fc11db0e","Type":"ContainerStarted","Data":"265a8ffae65f6a9674b1bb81f93f481d6bb2e80ef15626bf5520b4c1ff868f6d"}
Sep 11 11:58:23.451560 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:23.451535    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75" event={"ID":"281f2d78-83e8-47ab-8f2c-0ca48db6353e","Type":"ContainerStarted","Data":"49cf4ff0a149558a4f663610365873780f41ba4ded862bfab7f1427facfe17f2"}
Sep 11 11:58:23.451694 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:23.451583    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75" event={"ID":"281f2d78-83e8-47ab-8f2c-0ca48db6353e","Type":"ContainerStarted","Data":"a5620bbbf0ec83533d993e659f8abe9e704877fe9a7020fedcb7ae7fbef4c27e"}
Sep 11 11:58:23.451694 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:23.451622    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75"
Sep 11 11:58:23.468525 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:23.468475    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf" podStartSLOduration=1.468460509 podStartE2EDuration="1.468460509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:58:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:58:23.465973589 +0000 UTC m=+1443.886579617" watchObservedRunningTime="2026-09-11 11:58:23.468460509 +0000 UTC m=+1443.889066532"
Sep 11 11:58:23.484489 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:23.484440    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75" podStartSLOduration=1.484419376 podStartE2EDuration="1.484419376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:58:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:58:23.481148561 +0000 UTC m=+1443.901754587" watchObservedRunningTime="2026-09-11 11:58:23.484419376 +0000 UTC m=+1443.905025404"
Sep 11 11:58:31.437014 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:31.436981    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:34.459600 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:34.459573    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75"
Sep 11 11:58:34.460087 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:34.459988    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf"
Sep 11 11:58:35.280787 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:35.280753    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:42.632339 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:42.632306    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:45.438225 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:45.438187    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:58:48.479093 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:58:48.479061    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:00.152954 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:00.152914    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:01.145720 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:01.145680    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:01.784708 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:01.784672    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:03.965610 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:03.965562    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:03.988369 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:03.988344    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4"]
Sep 11 11:59:03.988611 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:03.988587    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4" podUID="e535cd14-2818-486e-b28e-2cfca1ac882a" containerName="payload-processing" containerID="cri-o://65650f7ed0127e2bdef4895a7f59ba160dd7801856081be28d8cfb88e9eddf25" gracePeriod=30
Sep 11 11:59:04.067086 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:04.067053    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:04.168794 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:04.168761    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:04.249180 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:04.249159    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4"
Sep 11 11:59:04.306128 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:04.306081    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:04.309269 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:04.309245    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e535cd14-2818-486e-b28e-2cfca1ac882a-plugins-config-volume\") pod \"e535cd14-2818-486e-b28e-2cfca1ac882a\" (UID: \"e535cd14-2818-486e-b28e-2cfca1ac882a\") "
Sep 11 11:59:04.309403 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:04.309373    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fj5tq\" (UniqueName: \"kubernetes.io/projected/e535cd14-2818-486e-b28e-2cfca1ac882a-kube-api-access-fj5tq\") pod \"e535cd14-2818-486e-b28e-2cfca1ac882a\" (UID: \"e535cd14-2818-486e-b28e-2cfca1ac882a\") "
Sep 11 11:59:04.309665 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:04.309621    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e535cd14-2818-486e-b28e-2cfca1ac882a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e535cd14-2818-486e-b28e-2cfca1ac882a" (UID: "e535cd14-2818-486e-b28e-2cfca1ac882a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:59:04.309771 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:04.309723    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e535cd14-2818-486e-b28e-2cfca1ac882a-plugins-config-volume\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:59:04.311548 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:04.311528    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e535cd14-2818-486e-b28e-2cfca1ac882a-kube-api-access-fj5tq" (OuterVolumeSpecName: "kube-api-access-fj5tq") pod "e535cd14-2818-486e-b28e-2cfca1ac882a" (UID: "e535cd14-2818-486e-b28e-2cfca1ac882a"). InnerVolumeSpecName "kube-api-access-fj5tq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:59:04.410734 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:04.410709    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fj5tq\" (UniqueName: \"kubernetes.io/projected/e535cd14-2818-486e-b28e-2cfca1ac882a-kube-api-access-fj5tq\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:59:04.587519 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:04.587490    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:04.669368 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:04.669339    2575 generic.go:356] "Generic (PLEG): container finished" podID="e535cd14-2818-486e-b28e-2cfca1ac882a" containerID="65650f7ed0127e2bdef4895a7f59ba160dd7801856081be28d8cfb88e9eddf25" exitCode=0
Sep 11 11:59:04.669520 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:04.669381    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4"
Sep 11 11:59:04.669520 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:04.669401    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4" event={"ID":"e535cd14-2818-486e-b28e-2cfca1ac882a","Type":"ContainerDied","Data":"65650f7ed0127e2bdef4895a7f59ba160dd7801856081be28d8cfb88e9eddf25"}
Sep 11 11:59:04.669520 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:04.669444    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4" event={"ID":"e535cd14-2818-486e-b28e-2cfca1ac882a","Type":"ContainerDied","Data":"50c9ec5530c2fb88d76fe915b910be9fc6aa6a821950cb3e573f846b16cd6790"}
Sep 11 11:59:04.669520 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:04.669467    2575 scope.go:117] "RemoveContainer" containerID="65650f7ed0127e2bdef4895a7f59ba160dd7801856081be28d8cfb88e9eddf25"
Sep 11 11:59:04.678822 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:04.678797    2575 scope.go:117] "RemoveContainer" containerID="65650f7ed0127e2bdef4895a7f59ba160dd7801856081be28d8cfb88e9eddf25"
Sep 11 11:59:04.679104 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:59:04.679084    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65650f7ed0127e2bdef4895a7f59ba160dd7801856081be28d8cfb88e9eddf25\": container with ID starting with 65650f7ed0127e2bdef4895a7f59ba160dd7801856081be28d8cfb88e9eddf25 not found: ID does not exist" containerID="65650f7ed0127e2bdef4895a7f59ba160dd7801856081be28d8cfb88e9eddf25"
Sep 11 11:59:04.679160 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:04.679113    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65650f7ed0127e2bdef4895a7f59ba160dd7801856081be28d8cfb88e9eddf25"} err="failed to get container status \"65650f7ed0127e2bdef4895a7f59ba160dd7801856081be28d8cfb88e9eddf25\": rpc error: code = NotFound desc = could not find container \"65650f7ed0127e2bdef4895a7f59ba160dd7801856081be28d8cfb88e9eddf25\": container with ID starting with 65650f7ed0127e2bdef4895a7f59ba160dd7801856081be28d8cfb88e9eddf25 not found: ID does not exist"
Sep 11 11:59:04.692051 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:04.692031    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4"]
Sep 11 11:59:04.695892 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:04.695872    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f9def1-5dc85f4c85-qfqj4"]
Sep 11 11:59:06.081807 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:06.081765    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e535cd14-2818-486e-b28e-2cfca1ac882a" path="/var/lib/kubelet/pods/e535cd14-2818-486e-b28e-2cfca1ac882a/volumes"
Sep 11 11:59:08.192370 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:08.192340    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:11.111176 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:11.111137    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:11.212451 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:11.212416    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:11.408082 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:11.407998    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:11.941558 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:11.941530    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:12.619393 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:12.619360    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:27.392893 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.392852    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:27.424787 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.424756    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75"]
Sep 11 11:59:27.425032 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.424986    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75" podUID="281f2d78-83e8-47ab-8f2c-0ca48db6353e" containerName="payload-processing" containerID="cri-o://49cf4ff0a149558a4f663610365873780f41ba4ded862bfab7f1427facfe17f2" gracePeriod=30
Sep 11 11:59:27.445615 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.445588    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf"]
Sep 11 11:59:27.445871 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.445848    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf" podUID="cc2df851-cac0-4266-9f57-32e4fc11db0e" containerName="payload-pre-processing" containerID="cri-o://6018dbe483a41dd6b41f3dff552f5b69742b05b44f0f09b00116a74a9a6c6bc5" gracePeriod=30
Sep 11 11:59:27.494253 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.494216    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:27.567161 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.567135    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:27.652758 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.652645    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:27.692899 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.692881    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75"
Sep 11 11:59:27.726738 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.726716    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf"
Sep 11 11:59:27.728883 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.728864    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/281f2d78-83e8-47ab-8f2c-0ca48db6353e-plugins-config-volume\") pod \"281f2d78-83e8-47ab-8f2c-0ca48db6353e\" (UID: \"281f2d78-83e8-47ab-8f2c-0ca48db6353e\") "
Sep 11 11:59:27.728999 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.728971    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sq6n7\" (UniqueName: \"kubernetes.io/projected/281f2d78-83e8-47ab-8f2c-0ca48db6353e-kube-api-access-sq6n7\") pod \"281f2d78-83e8-47ab-8f2c-0ca48db6353e\" (UID: \"281f2d78-83e8-47ab-8f2c-0ca48db6353e\") "
Sep 11 11:59:27.729176 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.729155    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/281f2d78-83e8-47ab-8f2c-0ca48db6353e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "281f2d78-83e8-47ab-8f2c-0ca48db6353e" (UID: "281f2d78-83e8-47ab-8f2c-0ca48db6353e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:59:27.729317 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.729304    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/281f2d78-83e8-47ab-8f2c-0ca48db6353e-plugins-config-volume\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:59:27.731213 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.731191    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/281f2d78-83e8-47ab-8f2c-0ca48db6353e-kube-api-access-sq6n7" (OuterVolumeSpecName: "kube-api-access-sq6n7") pod "281f2d78-83e8-47ab-8f2c-0ca48db6353e" (UID: "281f2d78-83e8-47ab-8f2c-0ca48db6353e"). InnerVolumeSpecName "kube-api-access-sq6n7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:59:27.764096 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.764068    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:27.808031 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.808004    2575 generic.go:356] "Generic (PLEG): container finished" podID="281f2d78-83e8-47ab-8f2c-0ca48db6353e" containerID="49cf4ff0a149558a4f663610365873780f41ba4ded862bfab7f1427facfe17f2" exitCode=0
Sep 11 11:59:27.808158 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.808049    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75"
Sep 11 11:59:27.808158 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.808091    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75" event={"ID":"281f2d78-83e8-47ab-8f2c-0ca48db6353e","Type":"ContainerDied","Data":"49cf4ff0a149558a4f663610365873780f41ba4ded862bfab7f1427facfe17f2"}
Sep 11 11:59:27.808158 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.808143    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75" event={"ID":"281f2d78-83e8-47ab-8f2c-0ca48db6353e","Type":"ContainerDied","Data":"a5620bbbf0ec83533d993e659f8abe9e704877fe9a7020fedcb7ae7fbef4c27e"}
Sep 11 11:59:27.808306 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.808172    2575 scope.go:117] "RemoveContainer" containerID="49cf4ff0a149558a4f663610365873780f41ba4ded862bfab7f1427facfe17f2"
Sep 11 11:59:27.809565 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.809544    2575 generic.go:356] "Generic (PLEG): container finished" podID="cc2df851-cac0-4266-9f57-32e4fc11db0e" containerID="6018dbe483a41dd6b41f3dff552f5b69742b05b44f0f09b00116a74a9a6c6bc5" exitCode=0
Sep 11 11:59:27.809697 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.809596    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf"
Sep 11 11:59:27.809697 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.809600    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf" event={"ID":"cc2df851-cac0-4266-9f57-32e4fc11db0e","Type":"ContainerDied","Data":"6018dbe483a41dd6b41f3dff552f5b69742b05b44f0f09b00116a74a9a6c6bc5"}
Sep 11 11:59:27.809697 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.809677    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf" event={"ID":"cc2df851-cac0-4266-9f57-32e4fc11db0e","Type":"ContainerDied","Data":"265a8ffae65f6a9674b1bb81f93f481d6bb2e80ef15626bf5520b4c1ff868f6d"}
Sep 11 11:59:27.817749 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.817727    2575 scope.go:117] "RemoveContainer" containerID="49cf4ff0a149558a4f663610365873780f41ba4ded862bfab7f1427facfe17f2"
Sep 11 11:59:27.818009 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:59:27.817984    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49cf4ff0a149558a4f663610365873780f41ba4ded862bfab7f1427facfe17f2\": container with ID starting with 49cf4ff0a149558a4f663610365873780f41ba4ded862bfab7f1427facfe17f2 not found: ID does not exist" containerID="49cf4ff0a149558a4f663610365873780f41ba4ded862bfab7f1427facfe17f2"
Sep 11 11:59:27.818124 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.818013    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49cf4ff0a149558a4f663610365873780f41ba4ded862bfab7f1427facfe17f2"} err="failed to get container status \"49cf4ff0a149558a4f663610365873780f41ba4ded862bfab7f1427facfe17f2\": rpc error: code = NotFound desc = could not find container \"49cf4ff0a149558a4f663610365873780f41ba4ded862bfab7f1427facfe17f2\": container with ID starting with 49cf4ff0a149558a4f663610365873780f41ba4ded862bfab7f1427facfe17f2 not found: ID does not exist"
Sep 11 11:59:27.818124 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.818029    2575 scope.go:117] "RemoveContainer" containerID="6018dbe483a41dd6b41f3dff552f5b69742b05b44f0f09b00116a74a9a6c6bc5"
Sep 11 11:59:27.826470 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.826453    2575 scope.go:117] "RemoveContainer" containerID="6018dbe483a41dd6b41f3dff552f5b69742b05b44f0f09b00116a74a9a6c6bc5"
Sep 11 11:59:27.826796 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:59:27.826778    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6018dbe483a41dd6b41f3dff552f5b69742b05b44f0f09b00116a74a9a6c6bc5\": container with ID starting with 6018dbe483a41dd6b41f3dff552f5b69742b05b44f0f09b00116a74a9a6c6bc5 not found: ID does not exist" containerID="6018dbe483a41dd6b41f3dff552f5b69742b05b44f0f09b00116a74a9a6c6bc5"
Sep 11 11:59:27.826865 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.826801    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6018dbe483a41dd6b41f3dff552f5b69742b05b44f0f09b00116a74a9a6c6bc5"} err="failed to get container status \"6018dbe483a41dd6b41f3dff552f5b69742b05b44f0f09b00116a74a9a6c6bc5\": rpc error: code = NotFound desc = could not find container \"6018dbe483a41dd6b41f3dff552f5b69742b05b44f0f09b00116a74a9a6c6bc5\": container with ID starting with 6018dbe483a41dd6b41f3dff552f5b69742b05b44f0f09b00116a74a9a6c6bc5 not found: ID does not exist"
Sep 11 11:59:27.830149 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.830128    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc2df851-cac0-4266-9f57-32e4fc11db0e-plugins-config-volume\") pod \"cc2df851-cac0-4266-9f57-32e4fc11db0e\" (UID: \"cc2df851-cac0-4266-9f57-32e4fc11db0e\") "
Sep 11 11:59:27.830256 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.830244    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdr8t\" (UniqueName: \"kubernetes.io/projected/cc2df851-cac0-4266-9f57-32e4fc11db0e-kube-api-access-tdr8t\") pod \"cc2df851-cac0-4266-9f57-32e4fc11db0e\" (UID: \"cc2df851-cac0-4266-9f57-32e4fc11db0e\") "
Sep 11 11:59:27.830524 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.830502    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc2df851-cac0-4266-9f57-32e4fc11db0e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cc2df851-cac0-4266-9f57-32e4fc11db0e" (UID: "cc2df851-cac0-4266-9f57-32e4fc11db0e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:59:27.830582 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.830529    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sq6n7\" (UniqueName: \"kubernetes.io/projected/281f2d78-83e8-47ab-8f2c-0ca48db6353e-kube-api-access-sq6n7\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:59:27.832473 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.832449    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc2df851-cac0-4266-9f57-32e4fc11db0e-kube-api-access-tdr8t" (OuterVolumeSpecName: "kube-api-access-tdr8t") pod "cc2df851-cac0-4266-9f57-32e4fc11db0e" (UID: "cc2df851-cac0-4266-9f57-32e4fc11db0e"). InnerVolumeSpecName "kube-api-access-tdr8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:59:27.834748 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.834725    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75"]
Sep 11 11:59:27.838932 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.838902    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-641847-5fd49b95f7-pxh75"]
Sep 11 11:59:27.931005 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.930949    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdr8t\" (UniqueName: \"kubernetes.io/projected/cc2df851-cac0-4266-9f57-32e4fc11db0e-kube-api-access-tdr8t\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:59:27.931005 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:27.930970    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc2df851-cac0-4266-9f57-32e4fc11db0e-plugins-config-volume\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:59:28.028696 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:28.028665    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:28.082382 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:28.082352    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="281f2d78-83e8-47ab-8f2c-0ca48db6353e" path="/var/lib/kubelet/pods/281f2d78-83e8-47ab-8f2c-0ca48db6353e/volumes"
Sep 11 11:59:28.125127 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:28.125104    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf"]
Sep 11 11:59:28.129545 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:28.129525    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-641847-7f998cb67d-4l5cf"]
Sep 11 11:59:30.081421 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:30.081388    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc2df851-cac0-4266-9f57-32e4fc11db0e" path="/var/lib/kubelet/pods/cc2df851-cac0-4266-9f57-32e4fc11db0e/volumes"
Sep 11 11:59:33.941263 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:33.941223    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:34.010674 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:34.010620    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:34.225121 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:34.225044    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:34.678208 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:34.678169    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:43.026329 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.026257    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"]
Sep 11 11:59:43.028685 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.026798    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc2df851-cac0-4266-9f57-32e4fc11db0e" containerName="payload-pre-processing"
Sep 11 11:59:43.028685 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.026812    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc2df851-cac0-4266-9f57-32e4fc11db0e" containerName="payload-pre-processing"
Sep 11 11:59:43.028685 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.026827    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e535cd14-2818-486e-b28e-2cfca1ac882a" containerName="payload-processing"
Sep 11 11:59:43.028685 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.026832    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e535cd14-2818-486e-b28e-2cfca1ac882a" containerName="payload-processing"
Sep 11 11:59:43.028685 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.026844    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="281f2d78-83e8-47ab-8f2c-0ca48db6353e" containerName="payload-processing"
Sep 11 11:59:43.028685 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.026850    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="281f2d78-83e8-47ab-8f2c-0ca48db6353e" containerName="payload-processing"
Sep 11 11:59:43.028685 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.026920    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e535cd14-2818-486e-b28e-2cfca1ac882a" containerName="payload-processing"
Sep 11 11:59:43.028685 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.026926    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc2df851-cac0-4266-9f57-32e4fc11db0e" containerName="payload-pre-processing"
Sep 11 11:59:43.028685 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.026935    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="281f2d78-83e8-47ab-8f2c-0ca48db6353e" containerName="payload-processing"
Sep 11 11:59:43.030123 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.030107    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 11:59:43.034008 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.033973    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-117711\"/\"kube-root-ca.crt\""
Sep 11 11:59:43.034008 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.033986    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-117711\"/\"rate-test-model-117711-kserve-self-signed-certs\""
Sep 11 11:59:43.034008 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.034007    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-117711\"/\"openshift-service-ca.crt\""
Sep 11 11:59:43.034262 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.034114    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-117711\"/\"default-dockercfg-ndglz\""
Sep 11 11:59:43.038693 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.038626    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"]
Sep 11 11:59:43.060144 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.060113    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:43.177611 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.177580    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-home\") pod \"rate-test-model-117711-kserve-7784457c47-67b45\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") " pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 11:59:43.177778 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.177655    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t277n\" (UniqueName: \"kubernetes.io/projected/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-kube-api-access-t277n\") pod \"rate-test-model-117711-kserve-7784457c47-67b45\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") " pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 11:59:43.177778 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.177695    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-dshm\") pod \"rate-test-model-117711-kserve-7784457c47-67b45\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") " pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 11:59:43.177778 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.177767    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-tls-certs\") pod \"rate-test-model-117711-kserve-7784457c47-67b45\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") " pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 11:59:43.177891 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.177828    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-tmp-dir\") pod \"rate-test-model-117711-kserve-7784457c47-67b45\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") " pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 11:59:43.177891 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.177856    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-model-cache\") pod \"rate-test-model-117711-kserve-7784457c47-67b45\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") " pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 11:59:43.278564 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.278463    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-tls-certs\") pod \"rate-test-model-117711-kserve-7784457c47-67b45\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") " pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 11:59:43.278564 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.278552    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-tmp-dir\") pod \"rate-test-model-117711-kserve-7784457c47-67b45\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") " pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 11:59:43.278832 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.278588    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-model-cache\") pod \"rate-test-model-117711-kserve-7784457c47-67b45\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") " pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 11:59:43.278832 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.278670    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-home\") pod \"rate-test-model-117711-kserve-7784457c47-67b45\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") " pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 11:59:43.278832 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.278720    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t277n\" (UniqueName: \"kubernetes.io/projected/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-kube-api-access-t277n\") pod \"rate-test-model-117711-kserve-7784457c47-67b45\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") " pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 11:59:43.278832 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.278770    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-dshm\") pod \"rate-test-model-117711-kserve-7784457c47-67b45\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") " pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 11:59:43.279057 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.279039    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-model-cache\") pod \"rate-test-model-117711-kserve-7784457c47-67b45\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") " pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 11:59:43.279288 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.279265    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-home\") pod \"rate-test-model-117711-kserve-7784457c47-67b45\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") " pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 11:59:43.279486 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.279309    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-tmp-dir\") pod \"rate-test-model-117711-kserve-7784457c47-67b45\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") " pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 11:59:43.281264 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.281242    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-dshm\") pod \"rate-test-model-117711-kserve-7784457c47-67b45\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") " pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 11:59:43.281917 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.281896    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-tls-certs\") pod \"rate-test-model-117711-kserve-7784457c47-67b45\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") " pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 11:59:43.289266 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.289243    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t277n\" (UniqueName: \"kubernetes.io/projected/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-kube-api-access-t277n\") pod \"rate-test-model-117711-kserve-7784457c47-67b45\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") " pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 11:59:43.342328 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.342300    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 11:59:43.617203 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.617179    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"]
Sep 11 11:59:43.619120 ip-10-0-135-16 kubenswrapper[2575]: W0911 11:59:43.619091    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d26ef6c_0a66_4eae_bfc1_f4f24f49d2d7.slice/crio-48e4ce53c9495e67464a1edfab2932e8fdb18ec1f1d22fbb2dbc39dd36bc1857 WatchSource:0}: Error finding container 48e4ce53c9495e67464a1edfab2932e8fdb18ec1f1d22fbb2dbc39dd36bc1857: Status 404 returned error can't find the container with id 48e4ce53c9495e67464a1edfab2932e8fdb18ec1f1d22fbb2dbc39dd36bc1857
Sep 11 11:59:43.897099 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.897010    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45" event={"ID":"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7","Type":"ContainerStarted","Data":"81367813540e9b13eec255018bd5c3913d36d85352f6f78b3b8f3eaffa917c19"}
Sep 11 11:59:43.897099 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.897067    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45" event={"ID":"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7","Type":"ContainerStarted","Data":"48e4ce53c9495e67464a1edfab2932e8fdb18ec1f1d22fbb2dbc39dd36bc1857"}
Sep 11 11:59:43.914135 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:43.914089    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45" podStartSLOduration=0.914069905 podStartE2EDuration="914.069905ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 11:59:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 11:59:43.91351118 +0000 UTC m=+1524.334117208" watchObservedRunningTime="2026-09-11 11:59:43.914069905 +0000 UTC m=+1524.334675935"
Sep 11 11:59:45.248373 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.248343    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"]
Sep 11 11:59:45.248777 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.248532    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj" podUID="ffd6ad89-a551-4650-bdf0-5a49ab808479" containerName="maas-api" containerID="cri-o://f8b38c6ad68c0d3003438b8703dfe043a71109a1cc653d71a6d51a9f3a0be409" gracePeriod=30
Sep 11 11:59:45.287223 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.287189    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:45.336150 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.336120    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8"]
Sep 11 11:59:45.422216 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.422183    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:45.510345 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.510291    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"
Sep 11 11:59:45.523336 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.523315    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:45.600271 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.600242    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffd6ad89-a551-4650-bdf0-5a49ab808479-metrics-certs\") pod \"ffd6ad89-a551-4650-bdf0-5a49ab808479\" (UID: \"ffd6ad89-a551-4650-bdf0-5a49ab808479\") "
Sep 11 11:59:45.600409 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.600347    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfzdl\" (UniqueName: \"kubernetes.io/projected/ffd6ad89-a551-4650-bdf0-5a49ab808479-kube-api-access-kfzdl\") pod \"ffd6ad89-a551-4650-bdf0-5a49ab808479\" (UID: \"ffd6ad89-a551-4650-bdf0-5a49ab808479\") "
Sep 11 11:59:45.600409 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.600399    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffd6ad89-a551-4650-bdf0-5a49ab808479-openshift-service-ca\") pod \"ffd6ad89-a551-4650-bdf0-5a49ab808479\" (UID: \"ffd6ad89-a551-4650-bdf0-5a49ab808479\") "
Sep 11 11:59:45.600537 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.600447    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffd6ad89-a551-4650-bdf0-5a49ab808479-maas-api-tls\") pod \"ffd6ad89-a551-4650-bdf0-5a49ab808479\" (UID: \"ffd6ad89-a551-4650-bdf0-5a49ab808479\") "
Sep 11 11:59:45.600832 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.600806    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffd6ad89-a551-4650-bdf0-5a49ab808479-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ffd6ad89-a551-4650-bdf0-5a49ab808479" (UID: "ffd6ad89-a551-4650-bdf0-5a49ab808479"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:59:45.602677 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.602655    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffd6ad89-a551-4650-bdf0-5a49ab808479-kube-api-access-kfzdl" (OuterVolumeSpecName: "kube-api-access-kfzdl") pod "ffd6ad89-a551-4650-bdf0-5a49ab808479" (UID: "ffd6ad89-a551-4650-bdf0-5a49ab808479"). InnerVolumeSpecName "kube-api-access-kfzdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:59:45.602975 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.602957    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffd6ad89-a551-4650-bdf0-5a49ab808479-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ffd6ad89-a551-4650-bdf0-5a49ab808479" (UID: "ffd6ad89-a551-4650-bdf0-5a49ab808479"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:59:45.603103 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.603084    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffd6ad89-a551-4650-bdf0-5a49ab808479-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ffd6ad89-a551-4650-bdf0-5a49ab808479" (UID: "ffd6ad89-a551-4650-bdf0-5a49ab808479"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 11:59:45.629215 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.629184    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:45.701484 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.701455    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfzdl\" (UniqueName: \"kubernetes.io/projected/ffd6ad89-a551-4650-bdf0-5a49ab808479-kube-api-access-kfzdl\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:59:45.701484 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.701483    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffd6ad89-a551-4650-bdf0-5a49ab808479-openshift-service-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:59:45.701661 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.701493    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffd6ad89-a551-4650-bdf0-5a49ab808479-maas-api-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:59:45.701661 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.701508    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffd6ad89-a551-4650-bdf0-5a49ab808479-metrics-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:59:45.729298 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.729265    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:45.916787 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.916696    2575 generic.go:356] "Generic (PLEG): container finished" podID="ffd6ad89-a551-4650-bdf0-5a49ab808479" containerID="f8b38c6ad68c0d3003438b8703dfe043a71109a1cc653d71a6d51a9f3a0be409" exitCode=0
Sep 11 11:59:45.916787 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.916757    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"
Sep 11 11:59:45.916787 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.916775    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj" event={"ID":"ffd6ad89-a551-4650-bdf0-5a49ab808479","Type":"ContainerDied","Data":"f8b38c6ad68c0d3003438b8703dfe043a71109a1cc653d71a6d51a9f3a0be409"}
Sep 11 11:59:45.917048 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.916849    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj" event={"ID":"ffd6ad89-a551-4650-bdf0-5a49ab808479","Type":"ContainerDied","Data":"d8494128069ef3ae5598d248bbdd473fbb6be3c12d6a9680d0a9e1ea08cb206b"}
Sep 11 11:59:45.917048 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.916866    2575 scope.go:117] "RemoveContainer" containerID="f8b38c6ad68c0d3003438b8703dfe043a71109a1cc653d71a6d51a9f3a0be409"
Sep 11 11:59:45.917164 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.917047    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8" podUID="8a09969a-0d59-4774-a095-73b6f81c038c" containerName="payload-processing" containerID="cri-o://78de7cf0edbebba4a32e6b5ef6b466e02f999e3f64fb0e5ffcd0ffa023c4b408" gracePeriod=30
Sep 11 11:59:45.927734 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.927716    2575 scope.go:117] "RemoveContainer" containerID="f8b38c6ad68c0d3003438b8703dfe043a71109a1cc653d71a6d51a9f3a0be409"
Sep 11 11:59:45.928015 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:59:45.927996    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8b38c6ad68c0d3003438b8703dfe043a71109a1cc653d71a6d51a9f3a0be409\": container with ID starting with f8b38c6ad68c0d3003438b8703dfe043a71109a1cc653d71a6d51a9f3a0be409 not found: ID does not exist" containerID="f8b38c6ad68c0d3003438b8703dfe043a71109a1cc653d71a6d51a9f3a0be409"
Sep 11 11:59:45.928071 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.928023    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8b38c6ad68c0d3003438b8703dfe043a71109a1cc653d71a6d51a9f3a0be409"} err="failed to get container status \"f8b38c6ad68c0d3003438b8703dfe043a71109a1cc653d71a6d51a9f3a0be409\": rpc error: code = NotFound desc = could not find container \"f8b38c6ad68c0d3003438b8703dfe043a71109a1cc653d71a6d51a9f3a0be409\": container with ID starting with f8b38c6ad68c0d3003438b8703dfe043a71109a1cc653d71a6d51a9f3a0be409 not found: ID does not exist"
Sep 11 11:59:45.985346 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.985313    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"]
Sep 11 11:59:45.995377 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:45.995347    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-8989e8-57f5b4f6d9-t72fj"]
Sep 11 11:59:46.056626 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:46.056596    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:46.083143 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:46.083117    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffd6ad89-a551-4650-bdf0-5a49ab808479" path="/var/lib/kubelet/pods/ffd6ad89-a551-4650-bdf0-5a49ab808479/volumes"
Sep 11 11:59:46.157646 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:46.157616    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8"
Sep 11 11:59:46.307773 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:46.307744    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxgvj\" (UniqueName: \"kubernetes.io/projected/8a09969a-0d59-4774-a095-73b6f81c038c-kube-api-access-vxgvj\") pod \"8a09969a-0d59-4774-a095-73b6f81c038c\" (UID: \"8a09969a-0d59-4774-a095-73b6f81c038c\") "
Sep 11 11:59:46.308194 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:46.307832    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a09969a-0d59-4774-a095-73b6f81c038c-plugins-config-volume\") pod \"8a09969a-0d59-4774-a095-73b6f81c038c\" (UID: \"8a09969a-0d59-4774-a095-73b6f81c038c\") "
Sep 11 11:59:46.308268 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:46.308222    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a09969a-0d59-4774-a095-73b6f81c038c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8a09969a-0d59-4774-a095-73b6f81c038c" (UID: "8a09969a-0d59-4774-a095-73b6f81c038c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 11:59:46.310004 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:46.309982    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a09969a-0d59-4774-a095-73b6f81c038c-kube-api-access-vxgvj" (OuterVolumeSpecName: "kube-api-access-vxgvj") pod "8a09969a-0d59-4774-a095-73b6f81c038c" (UID: "8a09969a-0d59-4774-a095-73b6f81c038c"). InnerVolumeSpecName "kube-api-access-vxgvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 11:59:46.408991 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:46.408968    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxgvj\" (UniqueName: \"kubernetes.io/projected/8a09969a-0d59-4774-a095-73b6f81c038c-kube-api-access-vxgvj\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:59:46.408991 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:46.408989    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a09969a-0d59-4774-a095-73b6f81c038c-plugins-config-volume\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 11:59:46.924180 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:46.924149    2575 generic.go:356] "Generic (PLEG): container finished" podID="8a09969a-0d59-4774-a095-73b6f81c038c" containerID="78de7cf0edbebba4a32e6b5ef6b466e02f999e3f64fb0e5ffcd0ffa023c4b408" exitCode=0
Sep 11 11:59:46.924383 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:46.924198    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8"
Sep 11 11:59:46.924383 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:46.924281    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8" event={"ID":"8a09969a-0d59-4774-a095-73b6f81c038c","Type":"ContainerDied","Data":"78de7cf0edbebba4a32e6b5ef6b466e02f999e3f64fb0e5ffcd0ffa023c4b408"}
Sep 11 11:59:46.924383 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:46.924342    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8" event={"ID":"8a09969a-0d59-4774-a095-73b6f81c038c","Type":"ContainerDied","Data":"6ffbf5b8b7f50b60fb7e5b0b91c4f78d57e9c7311af3f4b2f0d4a8ea2d5fa010"}
Sep 11 11:59:46.924559 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:46.924397    2575 scope.go:117] "RemoveContainer" containerID="78de7cf0edbebba4a32e6b5ef6b466e02f999e3f64fb0e5ffcd0ffa023c4b408"
Sep 11 11:59:46.933766 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:46.933750    2575 scope.go:117] "RemoveContainer" containerID="78de7cf0edbebba4a32e6b5ef6b466e02f999e3f64fb0e5ffcd0ffa023c4b408"
Sep 11 11:59:46.934023 ip-10-0-135-16 kubenswrapper[2575]: E0911 11:59:46.934002    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78de7cf0edbebba4a32e6b5ef6b466e02f999e3f64fb0e5ffcd0ffa023c4b408\": container with ID starting with 78de7cf0edbebba4a32e6b5ef6b466e02f999e3f64fb0e5ffcd0ffa023c4b408 not found: ID does not exist" containerID="78de7cf0edbebba4a32e6b5ef6b466e02f999e3f64fb0e5ffcd0ffa023c4b408"
Sep 11 11:59:46.934074 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:46.934027    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78de7cf0edbebba4a32e6b5ef6b466e02f999e3f64fb0e5ffcd0ffa023c4b408"} err="failed to get container status \"78de7cf0edbebba4a32e6b5ef6b466e02f999e3f64fb0e5ffcd0ffa023c4b408\": rpc error: code = NotFound desc = could not find container \"78de7cf0edbebba4a32e6b5ef6b466e02f999e3f64fb0e5ffcd0ffa023c4b408\": container with ID starting with 78de7cf0edbebba4a32e6b5ef6b466e02f999e3f64fb0e5ffcd0ffa023c4b408 not found: ID does not exist"
Sep 11 11:59:46.950688 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:46.950670    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8"]
Sep 11 11:59:46.954728 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:46.954707    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-8989e8-7fb967dc5f-mqhd8"]
Sep 11 11:59:48.082228 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:48.082187    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a09969a-0d59-4774-a095-73b6f81c038c" path="/var/lib/kubelet/pods/8a09969a-0d59-4774-a095-73b6f81c038c/volumes"
Sep 11 11:59:49.865530 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:49.865500    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:52.371729 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:52.371691    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:52.466316 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:52.466285    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:52.630430 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:52.630328    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:53.101594 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:53.101563    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 11:59:53.342832 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:53.342803    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 11:59:53.342994 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:53.342846    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 11:59:53.355614 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:53.355555    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 11:59:53.980910 ip-10-0-135-16 kubenswrapper[2575]: I0911 11:59:53.980883    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 12:00:00.189282 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:00.189253    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:03.498753 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.498722    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"]
Sep 11 12:00:03.499206 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.499008    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft" podUID="edf26460-8713-439a-bd02-f26f2f81f554" containerName="maas-api" containerID="cri-o://445e820b2a478646e289dc07ece90794bc0ace8d5097a9eed86dbddd2b4b8bb9" gracePeriod=30
Sep 11 12:00:03.547053 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.547022    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:03.567437 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.567412    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl"]
Sep 11 12:00:03.567691 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.567666    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl" podUID="0331ad11-878f-409a-8856-1c40b7d8c7a4" containerName="payload-processing" containerID="cri-o://568bb88e77bde93dafb385a30b7d641b548ba48f759632a117709ef6d8e994f4" gracePeriod=30
Sep 11 12:00:03.813563 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.813541    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"
Sep 11 12:00:03.847120 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.847100    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl"
Sep 11 12:00:03.875159 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.875133    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edf26460-8713-439a-bd02-f26f2f81f554-maas-api-tls\") pod \"edf26460-8713-439a-bd02-f26f2f81f554\" (UID: \"edf26460-8713-439a-bd02-f26f2f81f554\") "
Sep 11 12:00:03.875159 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.875162    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edf26460-8713-439a-bd02-f26f2f81f554-metrics-certs\") pod \"edf26460-8713-439a-bd02-f26f2f81f554\" (UID: \"edf26460-8713-439a-bd02-f26f2f81f554\") "
Sep 11 12:00:03.875379 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.875195    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edf26460-8713-439a-bd02-f26f2f81f554-openshift-service-ca\") pod \"edf26460-8713-439a-bd02-f26f2f81f554\" (UID: \"edf26460-8713-439a-bd02-f26f2f81f554\") "
Sep 11 12:00:03.875379 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.875226    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vpz8\" (UniqueName: \"kubernetes.io/projected/edf26460-8713-439a-bd02-f26f2f81f554-kube-api-access-6vpz8\") pod \"edf26460-8713-439a-bd02-f26f2f81f554\" (UID: \"edf26460-8713-439a-bd02-f26f2f81f554\") "
Sep 11 12:00:03.875379 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.875270    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0331ad11-878f-409a-8856-1c40b7d8c7a4-plugins-config-volume\") pod \"0331ad11-878f-409a-8856-1c40b7d8c7a4\" (UID: \"0331ad11-878f-409a-8856-1c40b7d8c7a4\") "
Sep 11 12:00:03.875379 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.875348    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czf6h\" (UniqueName: \"kubernetes.io/projected/0331ad11-878f-409a-8856-1c40b7d8c7a4-kube-api-access-czf6h\") pod \"0331ad11-878f-409a-8856-1c40b7d8c7a4\" (UID: \"0331ad11-878f-409a-8856-1c40b7d8c7a4\") "
Sep 11 12:00:03.875610 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.875581    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edf26460-8713-439a-bd02-f26f2f81f554-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "edf26460-8713-439a-bd02-f26f2f81f554" (UID: "edf26460-8713-439a-bd02-f26f2f81f554"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:00:03.875699 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.875664    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0331ad11-878f-409a-8856-1c40b7d8c7a4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0331ad11-878f-409a-8856-1c40b7d8c7a4" (UID: "0331ad11-878f-409a-8856-1c40b7d8c7a4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:00:03.875777 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.875751    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edf26460-8713-439a-bd02-f26f2f81f554-openshift-service-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:00:03.875837 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.875780    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0331ad11-878f-409a-8856-1c40b7d8c7a4-plugins-config-volume\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:00:03.877616 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.877593    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edf26460-8713-439a-bd02-f26f2f81f554-kube-api-access-6vpz8" (OuterVolumeSpecName: "kube-api-access-6vpz8") pod "edf26460-8713-439a-bd02-f26f2f81f554" (UID: "edf26460-8713-439a-bd02-f26f2f81f554"). InnerVolumeSpecName "kube-api-access-6vpz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:00:03.877616 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.877594    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edf26460-8713-439a-bd02-f26f2f81f554-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "edf26460-8713-439a-bd02-f26f2f81f554" (UID: "edf26460-8713-439a-bd02-f26f2f81f554"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:00:03.877760 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.877722    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0331ad11-878f-409a-8856-1c40b7d8c7a4-kube-api-access-czf6h" (OuterVolumeSpecName: "kube-api-access-czf6h") pod "0331ad11-878f-409a-8856-1c40b7d8c7a4" (UID: "0331ad11-878f-409a-8856-1c40b7d8c7a4"). InnerVolumeSpecName "kube-api-access-czf6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:00:03.877999 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.877981    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edf26460-8713-439a-bd02-f26f2f81f554-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "edf26460-8713-439a-bd02-f26f2f81f554" (UID: "edf26460-8713-439a-bd02-f26f2f81f554"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:00:03.917619 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.917595    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:03.976473 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.976454    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czf6h\" (UniqueName: \"kubernetes.io/projected/0331ad11-878f-409a-8856-1c40b7d8c7a4-kube-api-access-czf6h\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:00:03.976590 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.976474    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edf26460-8713-439a-bd02-f26f2f81f554-maas-api-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:00:03.976590 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.976487    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edf26460-8713-439a-bd02-f26f2f81f554-metrics-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:00:03.976590 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:03.976521    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vpz8\" (UniqueName: \"kubernetes.io/projected/edf26460-8713-439a-bd02-f26f2f81f554-kube-api-access-6vpz8\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:00:04.022469 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:04.022441    2575 generic.go:356] "Generic (PLEG): container finished" podID="0331ad11-878f-409a-8856-1c40b7d8c7a4" containerID="568bb88e77bde93dafb385a30b7d641b548ba48f759632a117709ef6d8e994f4" exitCode=0
Sep 11 12:00:04.022562 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:04.022484    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl"
Sep 11 12:00:04.022562 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:04.022527    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl" event={"ID":"0331ad11-878f-409a-8856-1c40b7d8c7a4","Type":"ContainerDied","Data":"568bb88e77bde93dafb385a30b7d641b548ba48f759632a117709ef6d8e994f4"}
Sep 11 12:00:04.022713 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:04.022573    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl" event={"ID":"0331ad11-878f-409a-8856-1c40b7d8c7a4","Type":"ContainerDied","Data":"2877c131aa2bb7883dddff8c10c8dfc269a80c3018f1d1188f29fcf652fe02e2"}
Sep 11 12:00:04.022713 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:04.022602    2575 scope.go:117] "RemoveContainer" containerID="568bb88e77bde93dafb385a30b7d641b548ba48f759632a117709ef6d8e994f4"
Sep 11 12:00:04.024050 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:04.024030    2575 generic.go:356] "Generic (PLEG): container finished" podID="edf26460-8713-439a-bd02-f26f2f81f554" containerID="445e820b2a478646e289dc07ece90794bc0ace8d5097a9eed86dbddd2b4b8bb9" exitCode=0
Sep 11 12:00:04.024136 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:04.024082    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"
Sep 11 12:00:04.024136 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:04.024097    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft" event={"ID":"edf26460-8713-439a-bd02-f26f2f81f554","Type":"ContainerDied","Data":"445e820b2a478646e289dc07ece90794bc0ace8d5097a9eed86dbddd2b4b8bb9"}
Sep 11 12:00:04.024136 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:04.024128    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft" event={"ID":"edf26460-8713-439a-bd02-f26f2f81f554","Type":"ContainerDied","Data":"b0e7bd384f34191d5d17463cce52dc35358b2692d73f68db239b7e2da5f34a00"}
Sep 11 12:00:04.032970 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:04.032944    2575 scope.go:117] "RemoveContainer" containerID="568bb88e77bde93dafb385a30b7d641b548ba48f759632a117709ef6d8e994f4"
Sep 11 12:00:04.033283 ip-10-0-135-16 kubenswrapper[2575]: E0911 12:00:04.033262    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"568bb88e77bde93dafb385a30b7d641b548ba48f759632a117709ef6d8e994f4\": container with ID starting with 568bb88e77bde93dafb385a30b7d641b548ba48f759632a117709ef6d8e994f4 not found: ID does not exist" containerID="568bb88e77bde93dafb385a30b7d641b548ba48f759632a117709ef6d8e994f4"
Sep 11 12:00:04.033388 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:04.033291    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"568bb88e77bde93dafb385a30b7d641b548ba48f759632a117709ef6d8e994f4"} err="failed to get container status \"568bb88e77bde93dafb385a30b7d641b548ba48f759632a117709ef6d8e994f4\": rpc error: code = NotFound desc = could not find container \"568bb88e77bde93dafb385a30b7d641b548ba48f759632a117709ef6d8e994f4\": container with ID starting with 568bb88e77bde93dafb385a30b7d641b548ba48f759632a117709ef6d8e994f4 not found: ID does not exist"
Sep 11 12:00:04.033388 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:04.033309    2575 scope.go:117] "RemoveContainer" containerID="445e820b2a478646e289dc07ece90794bc0ace8d5097a9eed86dbddd2b4b8bb9"
Sep 11 12:00:04.041435 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:04.041419    2575 scope.go:117] "RemoveContainer" containerID="445e820b2a478646e289dc07ece90794bc0ace8d5097a9eed86dbddd2b4b8bb9"
Sep 11 12:00:04.042149 ip-10-0-135-16 kubenswrapper[2575]: E0911 12:00:04.042128    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"445e820b2a478646e289dc07ece90794bc0ace8d5097a9eed86dbddd2b4b8bb9\": container with ID starting with 445e820b2a478646e289dc07ece90794bc0ace8d5097a9eed86dbddd2b4b8bb9 not found: ID does not exist" containerID="445e820b2a478646e289dc07ece90794bc0ace8d5097a9eed86dbddd2b4b8bb9"
Sep 11 12:00:04.042189 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:04.042155    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"445e820b2a478646e289dc07ece90794bc0ace8d5097a9eed86dbddd2b4b8bb9"} err="failed to get container status \"445e820b2a478646e289dc07ece90794bc0ace8d5097a9eed86dbddd2b4b8bb9\": rpc error: code = NotFound desc = could not find container \"445e820b2a478646e289dc07ece90794bc0ace8d5097a9eed86dbddd2b4b8bb9\": container with ID starting with 445e820b2a478646e289dc07ece90794bc0ace8d5097a9eed86dbddd2b4b8bb9 not found: ID does not exist"
Sep 11 12:00:04.049233 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:04.049213    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl"]
Sep 11 12:00:04.054379 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:04.054360    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6daa53-77c66d685f-v8jnl"]
Sep 11 12:00:04.068253 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:04.068229    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"]
Sep 11 12:00:04.082545 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:04.082521    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0331ad11-878f-409a-8856-1c40b7d8c7a4" path="/var/lib/kubelet/pods/0331ad11-878f-409a-8856-1c40b7d8c7a4/volumes"
Sep 11 12:00:04.082795 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:04.082783    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6daa53-74c4676bb5-84nft"]
Sep 11 12:00:04.128775 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:04.128755    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:04.641313 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:04.641283    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:06.081901 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:06.081872    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edf26460-8713-439a-bd02-f26f2f81f554" path="/var/lib/kubelet/pods/edf26460-8713-439a-bd02-f26f2f81f554/volumes"
Sep 11 12:00:10.229810 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:10.229775    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:10.331324 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:10.331295    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:10.497705 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:10.497667    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:25.546611 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:25.546573    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:26.341096 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:26.341059    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"]
Sep 11 12:00:26.364695 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:26.364663    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:26.836293 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:26.836251    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:27.931758 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:27.931725    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:28.790906 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:28.790868    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:29.014185 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:29.014156    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:30.600230 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:30.600179    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:30.773323 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:30.773293    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:31.741075 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.741038    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"]
Sep 11 12:00:31.741744 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.741723    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffd6ad89-a551-4650-bdf0-5a49ab808479" containerName="maas-api"
Sep 11 12:00:31.741744 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.741744    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffd6ad89-a551-4650-bdf0-5a49ab808479" containerName="maas-api"
Sep 11 12:00:31.741914 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.741756    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0331ad11-878f-409a-8856-1c40b7d8c7a4" containerName="payload-processing"
Sep 11 12:00:31.741914 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.741764    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0331ad11-878f-409a-8856-1c40b7d8c7a4" containerName="payload-processing"
Sep 11 12:00:31.741914 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.741804    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edf26460-8713-439a-bd02-f26f2f81f554" containerName="maas-api"
Sep 11 12:00:31.741914 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.741812    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="edf26460-8713-439a-bd02-f26f2f81f554" containerName="maas-api"
Sep 11 12:00:31.741914 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.741821    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a09969a-0d59-4774-a095-73b6f81c038c" containerName="payload-processing"
Sep 11 12:00:31.741914 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.741829    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a09969a-0d59-4774-a095-73b6f81c038c" containerName="payload-processing"
Sep 11 12:00:31.742260 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.741928    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0331ad11-878f-409a-8856-1c40b7d8c7a4" containerName="payload-processing"
Sep 11 12:00:31.742260 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.741942    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="edf26460-8713-439a-bd02-f26f2f81f554" containerName="maas-api"
Sep 11 12:00:31.742260 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.741953    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffd6ad89-a551-4650-bdf0-5a49ab808479" containerName="maas-api"
Sep 11 12:00:31.742260 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.741965    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a09969a-0d59-4774-a095-73b6f81c038c" containerName="payload-processing"
Sep 11 12:00:31.746668 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.746644    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"
Sep 11 12:00:31.749575 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.749550    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-b19164-serving-cert\""
Sep 11 12:00:31.758605 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.758578    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"]
Sep 11 12:00:31.788363 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.788334    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:31.848103 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.848073    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/186d630a-edbc-4637-a975-3398feb4cef9-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9\" (UID: \"186d630a-edbc-4637-a975-3398feb4cef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"
Sep 11 12:00:31.848276 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.848111    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/186d630a-edbc-4637-a975-3398feb4cef9-metrics-certs\") pod \"maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9\" (UID: \"186d630a-edbc-4637-a975-3398feb4cef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"
Sep 11 12:00:31.848276 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.848210    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhwbv\" (UniqueName: \"kubernetes.io/projected/186d630a-edbc-4637-a975-3398feb4cef9-kube-api-access-dhwbv\") pod \"maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9\" (UID: \"186d630a-edbc-4637-a975-3398feb4cef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"
Sep 11 12:00:31.848276 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.848268    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/186d630a-edbc-4637-a975-3398feb4cef9-maas-api-tls\") pod \"maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9\" (UID: \"186d630a-edbc-4637-a975-3398feb4cef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"
Sep 11 12:00:31.912986 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.912951    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:31.949347 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.949316    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhwbv\" (UniqueName: \"kubernetes.io/projected/186d630a-edbc-4637-a975-3398feb4cef9-kube-api-access-dhwbv\") pod \"maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9\" (UID: \"186d630a-edbc-4637-a975-3398feb4cef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"
Sep 11 12:00:31.949504 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.949374    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/186d630a-edbc-4637-a975-3398feb4cef9-maas-api-tls\") pod \"maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9\" (UID: \"186d630a-edbc-4637-a975-3398feb4cef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"
Sep 11 12:00:31.949504 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.949462    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/186d630a-edbc-4637-a975-3398feb4cef9-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9\" (UID: \"186d630a-edbc-4637-a975-3398feb4cef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"
Sep 11 12:00:31.949504 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.949495    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/186d630a-edbc-4637-a975-3398feb4cef9-metrics-certs\") pod \"maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9\" (UID: \"186d630a-edbc-4637-a975-3398feb4cef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"
Sep 11 12:00:31.949723 ip-10-0-135-16 kubenswrapper[2575]: E0911 12:00:31.949519    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-serving-cert: secret "maas-api-e2e-ipp-a-b19164-serving-cert" not found
Sep 11 12:00:31.949723 ip-10-0-135-16 kubenswrapper[2575]: E0911 12:00:31.949584    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/186d630a-edbc-4637-a975-3398feb4cef9-maas-api-tls podName:186d630a-edbc-4637-a975-3398feb4cef9 nodeName:}" failed. No retries permitted until 2026-09-11 12:00:32.449569086 +0000 UTC m=+1572.870175090 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/186d630a-edbc-4637-a975-3398feb4cef9-maas-api-tls") pod "maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9" (UID: "186d630a-edbc-4637-a975-3398feb4cef9") : secret "maas-api-e2e-ipp-a-b19164-serving-cert" not found
Sep 11 12:00:31.950122 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.950102    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/186d630a-edbc-4637-a975-3398feb4cef9-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9\" (UID: \"186d630a-edbc-4637-a975-3398feb4cef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"
Sep 11 12:00:31.952130 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.952109    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/186d630a-edbc-4637-a975-3398feb4cef9-metrics-certs\") pod \"maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9\" (UID: \"186d630a-edbc-4637-a975-3398feb4cef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"
Sep 11 12:00:31.961645 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:31.961613    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhwbv\" (UniqueName: \"kubernetes.io/projected/186d630a-edbc-4637-a975-3398feb4cef9-kube-api-access-dhwbv\") pod \"maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9\" (UID: \"186d630a-edbc-4637-a975-3398feb4cef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"
Sep 11 12:00:32.062217 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:32.062126    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:32.454365 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:32.454259    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/186d630a-edbc-4637-a975-3398feb4cef9-maas-api-tls\") pod \"maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9\" (UID: \"186d630a-edbc-4637-a975-3398feb4cef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"
Sep 11 12:00:32.456862 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:32.456837    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/186d630a-edbc-4637-a975-3398feb4cef9-maas-api-tls\") pod \"maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9\" (UID: \"186d630a-edbc-4637-a975-3398feb4cef9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"
Sep 11 12:00:32.658706 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:32.658674    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"
Sep 11 12:00:32.665308 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:32.665285    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:32.794369 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:32.794333    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:32.858524 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:32.858490    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4"]
Sep 11 12:00:32.871085 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:32.871054    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4"
Sep 11 12:00:32.874816 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:32.874796    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-b19164-dockercfg-8mldx\""
Sep 11 12:00:32.874958 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:32.874822    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-b19164\""
Sep 11 12:00:32.881599 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:32.881518    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4"]
Sep 11 12:00:32.952111 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:32.952089    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"]
Sep 11 12:00:32.954696 ip-10-0-135-16 kubenswrapper[2575]: W0911 12:00:32.954667    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod186d630a_edbc_4637_a975_3398feb4cef9.slice/crio-741b56861df14ba008ae8f43bd23ee850f3c1e2e29e1da7d7193873cc9556ff0 WatchSource:0}: Error finding container 741b56861df14ba008ae8f43bd23ee850f3c1e2e29e1da7d7193873cc9556ff0: Status 404 returned error can't find the container with id 741b56861df14ba008ae8f43bd23ee850f3c1e2e29e1da7d7193873cc9556ff0
Sep 11 12:00:32.956547 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:32.956529    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 12:00:32.957343 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:32.957315    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmpf9\" (UniqueName: \"kubernetes.io/projected/20b52758-5ee3-432b-a6ef-3cc175d5df4d-kube-api-access-xmpf9\") pod \"payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4\" (UID: \"20b52758-5ee3-432b-a6ef-3cc175d5df4d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4"
Sep 11 12:00:32.957467 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:32.957396    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20b52758-5ee3-432b-a6ef-3cc175d5df4d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4\" (UID: \"20b52758-5ee3-432b-a6ef-3cc175d5df4d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4"
Sep 11 12:00:33.058384 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:33.058343    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmpf9\" (UniqueName: \"kubernetes.io/projected/20b52758-5ee3-432b-a6ef-3cc175d5df4d-kube-api-access-xmpf9\") pod \"payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4\" (UID: \"20b52758-5ee3-432b-a6ef-3cc175d5df4d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4"
Sep 11 12:00:33.058562 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:33.058412    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20b52758-5ee3-432b-a6ef-3cc175d5df4d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4\" (UID: \"20b52758-5ee3-432b-a6ef-3cc175d5df4d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4"
Sep 11 12:00:33.059104 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:33.059085    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20b52758-5ee3-432b-a6ef-3cc175d5df4d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4\" (UID: \"20b52758-5ee3-432b-a6ef-3cc175d5df4d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4"
Sep 11 12:00:33.068752 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:33.068731    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmpf9\" (UniqueName: \"kubernetes.io/projected/20b52758-5ee3-432b-a6ef-3cc175d5df4d-kube-api-access-xmpf9\") pod \"payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4\" (UID: \"20b52758-5ee3-432b-a6ef-3cc175d5df4d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4"
Sep 11 12:00:33.169452 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:33.169417    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9" event={"ID":"186d630a-edbc-4637-a975-3398feb4cef9","Type":"ContainerStarted","Data":"741b56861df14ba008ae8f43bd23ee850f3c1e2e29e1da7d7193873cc9556ff0"}
Sep 11 12:00:33.183834 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:33.183803    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4"
Sep 11 12:00:33.462107 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:33.462080    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4"]
Sep 11 12:00:33.464903 ip-10-0-135-16 kubenswrapper[2575]: W0911 12:00:33.464876    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20b52758_5ee3_432b_a6ef_3cc175d5df4d.slice/crio-e419978fe60104d56a94fcdc4161d7c7cb4d64e5da8dcb4bb1ba359f5e595bde WatchSource:0}: Error finding container e419978fe60104d56a94fcdc4161d7c7cb4d64e5da8dcb4bb1ba359f5e595bde: Status 404 returned error can't find the container with id e419978fe60104d56a94fcdc4161d7c7cb4d64e5da8dcb4bb1ba359f5e595bde
Sep 11 12:00:34.175854 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:34.175752    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9" event={"ID":"186d630a-edbc-4637-a975-3398feb4cef9","Type":"ContainerStarted","Data":"e83ae1d49a5e6095e5dc185cd12b4aa570a700f6a6e71b328f3f81c60ffbc268"}
Sep 11 12:00:34.175854 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:34.175851    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"
Sep 11 12:00:34.177380 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:34.177358    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4" event={"ID":"20b52758-5ee3-432b-a6ef-3cc175d5df4d","Type":"ContainerStarted","Data":"d73248329c9b63e91632ed45f810a55945d345bf36bdfd4d776e4d5a6a36e8f3"}
Sep 11 12:00:34.177506 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:34.177399    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4" event={"ID":"20b52758-5ee3-432b-a6ef-3cc175d5df4d","Type":"ContainerStarted","Data":"e419978fe60104d56a94fcdc4161d7c7cb4d64e5da8dcb4bb1ba359f5e595bde"}
Sep 11 12:00:34.177506 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:34.177424    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4"
Sep 11 12:00:34.202261 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:34.202214    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9" podStartSLOduration=2.426572819 podStartE2EDuration="3.202197592s" totalImagesPullingTime="775.624773ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:00:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:00:34.199824894 +0000 UTC m=+1574.620430920" watchObservedRunningTime="2026-09-11 12:00:34.202197592 +0000 UTC m=+1574.622803619"
Sep 11 12:00:34.224690 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:34.224608    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4" podStartSLOduration=2.224593938 podStartE2EDuration="2.224593938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:00:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:00:34.222167755 +0000 UTC m=+1574.642773783" watchObservedRunningTime="2026-09-11 12:00:34.224593938 +0000 UTC m=+1574.645199965"
Sep 11 12:00:40.188004 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:40.187972    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"
Sep 11 12:00:41.360280 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:41.360236    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45" podUID="7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7" containerName="main" containerID="cri-o://81367813540e9b13eec255018bd5c3913d36d85352f6f78b3b8f3eaffa917c19" gracePeriod=45
Sep 11 12:00:41.611903 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:41.611847    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 12:00:41.639850 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:41.639823    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t277n\" (UniqueName: \"kubernetes.io/projected/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-kube-api-access-t277n\") pod \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") "
Sep 11 12:00:41.639991 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:41.639866    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-tmp-dir\") pod \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") "
Sep 11 12:00:41.639991 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:41.639902    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-tls-certs\") pod \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") "
Sep 11 12:00:41.640105 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:41.639993    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-model-cache\") pod \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") "
Sep 11 12:00:41.640105 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:41.640044    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-home\") pod \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") "
Sep 11 12:00:41.640105 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:41.640097    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-dshm\") pod \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\" (UID: \"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7\") "
Sep 11 12:00:41.640263 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:41.640154    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7" (UID: "7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:00:41.640323 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:41.640273    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-model-cache" (OuterVolumeSpecName: "model-cache") pod "7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7" (UID: "7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:00:41.640384 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:41.640313    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-home" (OuterVolumeSpecName: "home") pod "7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7" (UID: "7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:00:41.640523 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:41.640505    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-home\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:00:41.640581 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:41.640528    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-tmp-dir\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:00:41.640581 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:41.640541    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-model-cache\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:00:41.642123 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:41.642100    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7" (UID: "7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:00:41.642220 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:41.642166    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-kube-api-access-t277n" (OuterVolumeSpecName: "kube-api-access-t277n") pod "7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7" (UID: "7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7"). InnerVolumeSpecName "kube-api-access-t277n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:00:41.642220 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:41.642189    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-dshm" (OuterVolumeSpecName: "dshm") pod "7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7" (UID: "7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:00:41.742032 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:41.741989    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-dshm\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:00:41.742032 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:41.742025    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t277n\" (UniqueName: \"kubernetes.io/projected/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-kube-api-access-t277n\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:00:41.742032 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:41.742037    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7-tls-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:00:42.224255 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:42.224218    2575 generic.go:356] "Generic (PLEG): container finished" podID="7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7" containerID="81367813540e9b13eec255018bd5c3913d36d85352f6f78b3b8f3eaffa917c19" exitCode=0
Sep 11 12:00:42.224424 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:42.224273    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"
Sep 11 12:00:42.224424 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:42.224303    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45" event={"ID":"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7","Type":"ContainerDied","Data":"81367813540e9b13eec255018bd5c3913d36d85352f6f78b3b8f3eaffa917c19"}
Sep 11 12:00:42.224424 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:42.224355    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45" event={"ID":"7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7","Type":"ContainerDied","Data":"48e4ce53c9495e67464a1edfab2932e8fdb18ec1f1d22fbb2dbc39dd36bc1857"}
Sep 11 12:00:42.224424 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:42.224377    2575 scope.go:117] "RemoveContainer" containerID="81367813540e9b13eec255018bd5c3913d36d85352f6f78b3b8f3eaffa917c19"
Sep 11 12:00:42.233712 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:42.233696    2575 scope.go:117] "RemoveContainer" containerID="81367813540e9b13eec255018bd5c3913d36d85352f6f78b3b8f3eaffa917c19"
Sep 11 12:00:42.234001 ip-10-0-135-16 kubenswrapper[2575]: E0911 12:00:42.233985    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81367813540e9b13eec255018bd5c3913d36d85352f6f78b3b8f3eaffa917c19\": container with ID starting with 81367813540e9b13eec255018bd5c3913d36d85352f6f78b3b8f3eaffa917c19 not found: ID does not exist" containerID="81367813540e9b13eec255018bd5c3913d36d85352f6f78b3b8f3eaffa917c19"
Sep 11 12:00:42.234116 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:42.234008    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81367813540e9b13eec255018bd5c3913d36d85352f6f78b3b8f3eaffa917c19"} err="failed to get container status \"81367813540e9b13eec255018bd5c3913d36d85352f6f78b3b8f3eaffa917c19\": rpc error: code = NotFound desc = could not find container \"81367813540e9b13eec255018bd5c3913d36d85352f6f78b3b8f3eaffa917c19\": container with ID starting with 81367813540e9b13eec255018bd5c3913d36d85352f6f78b3b8f3eaffa917c19 not found: ID does not exist"
Sep 11 12:00:42.244736 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:42.244710    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"]
Sep 11 12:00:42.249853 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:42.249831    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-117711/rate-test-model-117711-kserve-7784457c47-67b45"]
Sep 11 12:00:44.082242 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:44.082211    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7" path="/var/lib/kubelet/pods/7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7/volumes"
Sep 11 12:00:45.184959 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:45.184934    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4"
Sep 11 12:00:55.451364 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:55.451328    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:56.078249 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:56.078218    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:56.746803 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:56.746765    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:57.198209 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:57.198120    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:59.179640 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:59.179604    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:00:59.286075 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:00:59.286044    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:00.434582 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:00.434547    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:00.536437 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:00.536404    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:00.654437 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:00.654401    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:00.992090 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:00.992052    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:01.103468 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.103435    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl"]
Sep 11 12:01:01.103918 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.103899    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7" containerName="main"
Sep 11 12:01:01.103918 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.103918    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7" containerName="main"
Sep 11 12:01:01.104074 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.104007    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d26ef6c-0a66-4eae-bfc1-f4f24f49d2d7" containerName="main"
Sep 11 12:01:01.107046 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.107030    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl"
Sep 11 12:01:01.110238 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.110219    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-720006-dockercfg-mj4mf\""
Sep 11 12:01:01.110337 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.110246    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-720006\""
Sep 11 12:01:01.128059 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.128028    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl"]
Sep 11 12:01:01.135690 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.135661    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x"]
Sep 11 12:01:01.139202 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.139185    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x"
Sep 11 12:01:01.150956 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.150936    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x"]
Sep 11 12:01:01.172321 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.172296    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:01.231444 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.231413    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b384fe0-a67d-4ce5-b58d-5277bd7b5770-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x\" (UID: \"2b384fe0-a67d-4ce5-b58d-5277bd7b5770\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x"
Sep 11 12:01:01.231601 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.231460    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9d56\" (UniqueName: \"kubernetes.io/projected/d7eb0d7c-f90e-42e5-a142-9a640b32683b-kube-api-access-x9d56\") pod \"payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl\" (UID: \"d7eb0d7c-f90e-42e5-a142-9a640b32683b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl"
Sep 11 12:01:01.231601 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.231583    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7eb0d7c-f90e-42e5-a142-9a640b32683b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl\" (UID: \"d7eb0d7c-f90e-42e5-a142-9a640b32683b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl"
Sep 11 12:01:01.231720 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.231610    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx2f8\" (UniqueName: \"kubernetes.io/projected/2b384fe0-a67d-4ce5-b58d-5277bd7b5770-kube-api-access-qx2f8\") pod \"payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x\" (UID: \"2b384fe0-a67d-4ce5-b58d-5277bd7b5770\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x"
Sep 11 12:01:01.332504 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.332424    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7eb0d7c-f90e-42e5-a142-9a640b32683b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl\" (UID: \"d7eb0d7c-f90e-42e5-a142-9a640b32683b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl"
Sep 11 12:01:01.332504 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.332465    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx2f8\" (UniqueName: \"kubernetes.io/projected/2b384fe0-a67d-4ce5-b58d-5277bd7b5770-kube-api-access-qx2f8\") pod \"payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x\" (UID: \"2b384fe0-a67d-4ce5-b58d-5277bd7b5770\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x"
Sep 11 12:01:01.332770 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.332508    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b384fe0-a67d-4ce5-b58d-5277bd7b5770-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x\" (UID: \"2b384fe0-a67d-4ce5-b58d-5277bd7b5770\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x"
Sep 11 12:01:01.332770 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.332540    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9d56\" (UniqueName: \"kubernetes.io/projected/d7eb0d7c-f90e-42e5-a142-9a640b32683b-kube-api-access-x9d56\") pod \"payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl\" (UID: \"d7eb0d7c-f90e-42e5-a142-9a640b32683b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl"
Sep 11 12:01:01.333265 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.333242    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7eb0d7c-f90e-42e5-a142-9a640b32683b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl\" (UID: \"d7eb0d7c-f90e-42e5-a142-9a640b32683b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl"
Sep 11 12:01:01.333365 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.333251    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b384fe0-a67d-4ce5-b58d-5277bd7b5770-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x\" (UID: \"2b384fe0-a67d-4ce5-b58d-5277bd7b5770\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x"
Sep 11 12:01:01.341277 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.341245    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx2f8\" (UniqueName: \"kubernetes.io/projected/2b384fe0-a67d-4ce5-b58d-5277bd7b5770-kube-api-access-qx2f8\") pod \"payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x\" (UID: \"2b384fe0-a67d-4ce5-b58d-5277bd7b5770\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x"
Sep 11 12:01:01.341372 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.341326    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9d56\" (UniqueName: \"kubernetes.io/projected/d7eb0d7c-f90e-42e5-a142-9a640b32683b-kube-api-access-x9d56\") pod \"payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl\" (UID: \"d7eb0d7c-f90e-42e5-a142-9a640b32683b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl"
Sep 11 12:01:01.417164 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.417127    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl"
Sep 11 12:01:01.450456 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.450428    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x"
Sep 11 12:01:01.694890 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.694861    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl"]
Sep 11 12:01:01.696536 ip-10-0-135-16 kubenswrapper[2575]: W0911 12:01:01.696510    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7eb0d7c_f90e_42e5_a142_9a640b32683b.slice/crio-3ab8df467130804e63da9fef43a46c2edfea2f47b0cfd5be437e863b38a36413 WatchSource:0}: Error finding container 3ab8df467130804e63da9fef43a46c2edfea2f47b0cfd5be437e863b38a36413: Status 404 returned error can't find the container with id 3ab8df467130804e63da9fef43a46c2edfea2f47b0cfd5be437e863b38a36413
Sep 11 12:01:01.735177 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:01.735153    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x"]
Sep 11 12:01:01.736783 ip-10-0-135-16 kubenswrapper[2575]: W0911 12:01:01.736756    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b384fe0_a67d_4ce5_b58d_5277bd7b5770.slice/crio-40143dad78f2199951ba9be40f37ad9a896ce4b7a784161ef24085d055af1577 WatchSource:0}: Error finding container 40143dad78f2199951ba9be40f37ad9a896ce4b7a784161ef24085d055af1577: Status 404 returned error can't find the container with id 40143dad78f2199951ba9be40f37ad9a896ce4b7a784161ef24085d055af1577
Sep 11 12:01:02.330502 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:02.330456    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x" event={"ID":"2b384fe0-a67d-4ce5-b58d-5277bd7b5770","Type":"ContainerStarted","Data":"12d2ca9f2b437cd6f7f2467e213012de38229fb31217ebf1a8ba414aa5a534f1"}
Sep 11 12:01:02.330731 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:02.330523    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x" event={"ID":"2b384fe0-a67d-4ce5-b58d-5277bd7b5770","Type":"ContainerStarted","Data":"40143dad78f2199951ba9be40f37ad9a896ce4b7a784161ef24085d055af1577"}
Sep 11 12:01:02.330731 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:02.330549    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x"
Sep 11 12:01:02.332126 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:02.332096    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl" event={"ID":"d7eb0d7c-f90e-42e5-a142-9a640b32683b","Type":"ContainerStarted","Data":"995eccf69b507f3bb0856091568856067b8fb98c5b2f96723ab18d543dad1a81"}
Sep 11 12:01:02.332259 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:02.332145    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl" event={"ID":"d7eb0d7c-f90e-42e5-a142-9a640b32683b","Type":"ContainerStarted","Data":"3ab8df467130804e63da9fef43a46c2edfea2f47b0cfd5be437e863b38a36413"}
Sep 11 12:01:02.332259 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:02.332174    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl"
Sep 11 12:01:02.348862 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:02.348821    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x" podStartSLOduration=1.348804901 podStartE2EDuration="1.348804901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:01:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:01:02.34747148 +0000 UTC m=+1602.768077508" watchObservedRunningTime="2026-09-11 12:01:02.348804901 +0000 UTC m=+1602.769410977"
Sep 11 12:01:02.365741 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:02.365699    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl" podStartSLOduration=1.365683982 podStartE2EDuration="1.365683982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:01:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:01:02.364532662 +0000 UTC m=+1602.785138694" watchObservedRunningTime="2026-09-11 12:01:02.365683982 +0000 UTC m=+1602.786290009"
Sep 11 12:01:13.338876 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:13.338793    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x"
Sep 11 12:01:13.338876 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:13.338872    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl"
Sep 11 12:01:24.154508 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:24.154468    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:29.016865 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:29.016801    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qqpzb_66d1dfc4-81b5-49fc-bfa7-5c023917525a/payload-processing/0.log"
Sep 11 12:01:29.133463 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:29.133430    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x_2b384fe0-a67d-4ce5-b58d-5277bd7b5770/payload-processing/0.log"
Sep 11 12:01:36.321020 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:36.320993    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:40.622926 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:40.622890    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:40.723766 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:40.723732    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:01:59.639305 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:59.639272    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qqpzb_66d1dfc4-81b5-49fc-bfa7-5c023917525a/payload-processing/0.log"
Sep 11 12:01:59.751313 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:01:59.751284    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x_2b384fe0-a67d-4ce5-b58d-5277bd7b5770/payload-processing/0.log"
Sep 11 12:02:00.021090 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:00.021060    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:00.393121 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:00.393033    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:00.817356 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:00.817327    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:04.009337 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:04.009296    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:04.078788 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:04.078757    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:04.975887 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:04.975852    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"]
Sep 11 12:02:04.979907 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:04.979884    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"
Sep 11 12:02:04.983355 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:04.983327    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-b8e0f0-serving-cert\""
Sep 11 12:02:04.995976 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:04.995945    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"]
Sep 11 12:02:05.048217 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:05.048185    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/007ae1ce-fc4f-4ef9-944b-432df2c76108-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc\" (UID: \"007ae1ce-fc4f-4ef9-944b-432df2c76108\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"
Sep 11 12:02:05.048730 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:05.048244    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/007ae1ce-fc4f-4ef9-944b-432df2c76108-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc\" (UID: \"007ae1ce-fc4f-4ef9-944b-432df2c76108\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"
Sep 11 12:02:05.048730 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:05.048412    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/007ae1ce-fc4f-4ef9-944b-432df2c76108-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc\" (UID: \"007ae1ce-fc4f-4ef9-944b-432df2c76108\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"
Sep 11 12:02:05.048730 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:05.048463    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nxm8\" (UniqueName: \"kubernetes.io/projected/007ae1ce-fc4f-4ef9-944b-432df2c76108-kube-api-access-7nxm8\") pod \"maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc\" (UID: \"007ae1ce-fc4f-4ef9-944b-432df2c76108\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"
Sep 11 12:02:05.058430 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:05.058405    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:05.149862 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:05.149831    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/007ae1ce-fc4f-4ef9-944b-432df2c76108-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc\" (UID: \"007ae1ce-fc4f-4ef9-944b-432df2c76108\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"
Sep 11 12:02:05.149862 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:05.149867    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nxm8\" (UniqueName: \"kubernetes.io/projected/007ae1ce-fc4f-4ef9-944b-432df2c76108-kube-api-access-7nxm8\") pod \"maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc\" (UID: \"007ae1ce-fc4f-4ef9-944b-432df2c76108\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"
Sep 11 12:02:05.150121 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:05.149922    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/007ae1ce-fc4f-4ef9-944b-432df2c76108-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc\" (UID: \"007ae1ce-fc4f-4ef9-944b-432df2c76108\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"
Sep 11 12:02:05.150121 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:05.149971    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/007ae1ce-fc4f-4ef9-944b-432df2c76108-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc\" (UID: \"007ae1ce-fc4f-4ef9-944b-432df2c76108\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"
Sep 11 12:02:05.150121 ip-10-0-135-16 kubenswrapper[2575]: E0911 12:02:05.150063    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-serving-cert: secret "maas-api-e2e-ipp-cleanup-b8e0f0-serving-cert" not found
Sep 11 12:02:05.150269 ip-10-0-135-16 kubenswrapper[2575]: E0911 12:02:05.150144    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/007ae1ce-fc4f-4ef9-944b-432df2c76108-maas-api-tls podName:007ae1ce-fc4f-4ef9-944b-432df2c76108 nodeName:}" failed. No retries permitted until 2026-09-11 12:02:05.650121712 +0000 UTC m=+1666.070727732 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/007ae1ce-fc4f-4ef9-944b-432df2c76108-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc" (UID: "007ae1ce-fc4f-4ef9-944b-432df2c76108") : secret "maas-api-e2e-ipp-cleanup-b8e0f0-serving-cert" not found
Sep 11 12:02:05.150908 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:05.150868    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/007ae1ce-fc4f-4ef9-944b-432df2c76108-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc\" (UID: \"007ae1ce-fc4f-4ef9-944b-432df2c76108\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"
Sep 11 12:02:05.152781 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:05.152754    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/007ae1ce-fc4f-4ef9-944b-432df2c76108-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc\" (UID: \"007ae1ce-fc4f-4ef9-944b-432df2c76108\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"
Sep 11 12:02:05.172675 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:05.172649    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:05.174678 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:05.174624    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nxm8\" (UniqueName: \"kubernetes.io/projected/007ae1ce-fc4f-4ef9-944b-432df2c76108-kube-api-access-7nxm8\") pod \"maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc\" (UID: \"007ae1ce-fc4f-4ef9-944b-432df2c76108\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"
Sep 11 12:02:05.317462 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:05.317426    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:05.653749 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:05.653662    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/007ae1ce-fc4f-4ef9-944b-432df2c76108-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc\" (UID: \"007ae1ce-fc4f-4ef9-944b-432df2c76108\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"
Sep 11 12:02:05.656262 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:05.656235    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/007ae1ce-fc4f-4ef9-944b-432df2c76108-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc\" (UID: \"007ae1ce-fc4f-4ef9-944b-432df2c76108\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"
Sep 11 12:02:05.891949 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:05.891917    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"
Sep 11 12:02:05.916758 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:05.916690    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:06.071750 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:06.071722    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:06.182239 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:06.182164    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b"]
Sep 11 12:02:06.186762 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:06.186745    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"]
Sep 11 12:02:06.186846 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:06.186835    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b"
Sep 11 12:02:06.188399 ip-10-0-135-16 kubenswrapper[2575]: W0911 12:02:06.188376    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod007ae1ce_fc4f_4ef9_944b_432df2c76108.slice/crio-0e2e8329221e1a3c3c05ec8c4fc613e7859bbd18e676f191534a9873f117b34e WatchSource:0}: Error finding container 0e2e8329221e1a3c3c05ec8c4fc613e7859bbd18e676f191534a9873f117b34e: Status 404 returned error can't find the container with id 0e2e8329221e1a3c3c05ec8c4fc613e7859bbd18e676f191534a9873f117b34e
Sep 11 12:02:06.192416 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:06.192383    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-b8e0f0\""
Sep 11 12:02:06.192416 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:06.192393    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-b8e0f0-dockercfg-4d9hd\""
Sep 11 12:02:06.207279 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:06.207257    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b"]
Sep 11 12:02:06.261075 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:06.261041    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r87x\" (UniqueName: \"kubernetes.io/projected/88d5dfba-057c-4513-a558-a9e8385361b9-kube-api-access-4r87x\") pod \"payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b\" (UID: \"88d5dfba-057c-4513-a558-a9e8385361b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b"
Sep 11 12:02:06.261258 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:06.261157    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88d5dfba-057c-4513-a558-a9e8385361b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b\" (UID: \"88d5dfba-057c-4513-a558-a9e8385361b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b"
Sep 11 12:02:06.361902 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:06.361871    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88d5dfba-057c-4513-a558-a9e8385361b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b\" (UID: \"88d5dfba-057c-4513-a558-a9e8385361b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b"
Sep 11 12:02:06.362073 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:06.361918    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r87x\" (UniqueName: \"kubernetes.io/projected/88d5dfba-057c-4513-a558-a9e8385361b9-kube-api-access-4r87x\") pod \"payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b\" (UID: \"88d5dfba-057c-4513-a558-a9e8385361b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b"
Sep 11 12:02:06.362513 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:06.362495    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88d5dfba-057c-4513-a558-a9e8385361b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b\" (UID: \"88d5dfba-057c-4513-a558-a9e8385361b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b"
Sep 11 12:02:06.372774 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:06.372748    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r87x\" (UniqueName: \"kubernetes.io/projected/88d5dfba-057c-4513-a558-a9e8385361b9-kube-api-access-4r87x\") pod \"payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b\" (UID: \"88d5dfba-057c-4513-a558-a9e8385361b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b"
Sep 11 12:02:06.509140 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:06.509107    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b"
Sep 11 12:02:06.678903 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:06.678868    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc" event={"ID":"007ae1ce-fc4f-4ef9-944b-432df2c76108","Type":"ContainerStarted","Data":"0e2e8329221e1a3c3c05ec8c4fc613e7859bbd18e676f191534a9873f117b34e"}
Sep 11 12:02:06.807204 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:06.807180    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b"]
Sep 11 12:02:06.809166 ip-10-0-135-16 kubenswrapper[2575]: W0911 12:02:06.809138    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88d5dfba_057c_4513_a558_a9e8385361b9.slice/crio-f51e336d32eabbaca9ad1644480c520176b5bba46ec8738e0b59be23d9c3a6da WatchSource:0}: Error finding container f51e336d32eabbaca9ad1644480c520176b5bba46ec8738e0b59be23d9c3a6da: Status 404 returned error can't find the container with id f51e336d32eabbaca9ad1644480c520176b5bba46ec8738e0b59be23d9c3a6da
Sep 11 12:02:07.686896 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:07.686858    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b" event={"ID":"88d5dfba-057c-4513-a558-a9e8385361b9","Type":"ContainerStarted","Data":"2b8adaec3587f0521a2d1ea07e25eda8b28ceb4f9cdd6688a9ff88e881999256"}
Sep 11 12:02:07.687382 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:07.686921    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b" event={"ID":"88d5dfba-057c-4513-a558-a9e8385361b9","Type":"ContainerStarted","Data":"f51e336d32eabbaca9ad1644480c520176b5bba46ec8738e0b59be23d9c3a6da"}
Sep 11 12:02:07.687382 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:07.686950    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b"
Sep 11 12:02:07.688546 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:07.688521    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc" event={"ID":"007ae1ce-fc4f-4ef9-944b-432df2c76108","Type":"ContainerStarted","Data":"fdee89962679d1aa4948c7111e78375ae11ceff640b2e8cbaa5dd225d6343bae"}
Sep 11 12:02:07.688704 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:07.688569    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"
Sep 11 12:02:07.710201 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:07.710154    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b" podStartSLOduration=1.710138231 podStartE2EDuration="1.710138231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:02:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:02:07.708522279 +0000 UTC m=+1668.129128307" watchObservedRunningTime="2026-09-11 12:02:07.710138231 +0000 UTC m=+1668.130744257"
Sep 11 12:02:07.740195 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:07.740147    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc" podStartSLOduration=2.798379906 podStartE2EDuration="3.740132496s" totalImagesPullingTime="941.75259ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:02:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:02:07.737670897 +0000 UTC m=+1668.158276921" watchObservedRunningTime="2026-09-11 12:02:07.740132496 +0000 UTC m=+1668.160738523"
Sep 11 12:02:13.699054 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:13.699021    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"
Sep 11 12:02:18.696776 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:18.696744    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b"
Sep 11 12:02:28.790275 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:28.790240    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:35.589488 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:35.589454    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"]
Sep 11 12:02:35.589963 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:35.589675    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc" podUID="007ae1ce-fc4f-4ef9-944b-432df2c76108" containerName="maas-api" containerID="cri-o://fdee89962679d1aa4948c7111e78375ae11ceff640b2e8cbaa5dd225d6343bae" gracePeriod=30
Sep 11 12:02:35.624007 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:35.623975    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:35.687620 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:35.687591    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b"]
Sep 11 12:02:35.687896 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:35.687867    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b" podUID="88d5dfba-057c-4513-a558-a9e8385361b9" containerName="payload-pre-processing" containerID="cri-o://2b8adaec3587f0521a2d1ea07e25eda8b28ceb4f9cdd6688a9ff88e881999256" gracePeriod=30
Sep 11 12:02:35.735111 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:35.735076    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:35.841876 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:35.841805    2575 generic.go:356] "Generic (PLEG): container finished" podID="88d5dfba-057c-4513-a558-a9e8385361b9" containerID="2b8adaec3587f0521a2d1ea07e25eda8b28ceb4f9cdd6688a9ff88e881999256" exitCode=0
Sep 11 12:02:35.841876 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:35.841858    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b" event={"ID":"88d5dfba-057c-4513-a558-a9e8385361b9","Type":"ContainerDied","Data":"2b8adaec3587f0521a2d1ea07e25eda8b28ceb4f9cdd6688a9ff88e881999256"}
Sep 11 12:02:35.843470 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:35.843442    2575 generic.go:356] "Generic (PLEG): container finished" podID="007ae1ce-fc4f-4ef9-944b-432df2c76108" containerID="fdee89962679d1aa4948c7111e78375ae11ceff640b2e8cbaa5dd225d6343bae" exitCode=0
Sep 11 12:02:35.843608 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:35.843499    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc" event={"ID":"007ae1ce-fc4f-4ef9-944b-432df2c76108","Type":"ContainerDied","Data":"fdee89962679d1aa4948c7111e78375ae11ceff640b2e8cbaa5dd225d6343bae"}
Sep 11 12:02:35.864016 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:35.863988    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"
Sep 11 12:02:35.873114 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:35.873085    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:35.946839 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:35.946813    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b"
Sep 11 12:02:35.953039 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:35.953017    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/007ae1ce-fc4f-4ef9-944b-432df2c76108-openshift-service-ca\") pod \"007ae1ce-fc4f-4ef9-944b-432df2c76108\" (UID: \"007ae1ce-fc4f-4ef9-944b-432df2c76108\") "
Sep 11 12:02:35.953168 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:35.953054    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nxm8\" (UniqueName: \"kubernetes.io/projected/007ae1ce-fc4f-4ef9-944b-432df2c76108-kube-api-access-7nxm8\") pod \"007ae1ce-fc4f-4ef9-944b-432df2c76108\" (UID: \"007ae1ce-fc4f-4ef9-944b-432df2c76108\") "
Sep 11 12:02:35.953168 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:35.953166    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/007ae1ce-fc4f-4ef9-944b-432df2c76108-metrics-certs\") pod \"007ae1ce-fc4f-4ef9-944b-432df2c76108\" (UID: \"007ae1ce-fc4f-4ef9-944b-432df2c76108\") "
Sep 11 12:02:35.953288 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:35.953195    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/007ae1ce-fc4f-4ef9-944b-432df2c76108-maas-api-tls\") pod \"007ae1ce-fc4f-4ef9-944b-432df2c76108\" (UID: \"007ae1ce-fc4f-4ef9-944b-432df2c76108\") "
Sep 11 12:02:35.953382 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:35.953361    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/007ae1ce-fc4f-4ef9-944b-432df2c76108-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "007ae1ce-fc4f-4ef9-944b-432df2c76108" (UID: "007ae1ce-fc4f-4ef9-944b-432df2c76108"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:02:35.953459 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:35.953445    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/007ae1ce-fc4f-4ef9-944b-432df2c76108-openshift-service-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:02:35.955344 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:35.955314    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/007ae1ce-fc4f-4ef9-944b-432df2c76108-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "007ae1ce-fc4f-4ef9-944b-432df2c76108" (UID: "007ae1ce-fc4f-4ef9-944b-432df2c76108"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:02:35.955344 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:35.955321    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/007ae1ce-fc4f-4ef9-944b-432df2c76108-kube-api-access-7nxm8" (OuterVolumeSpecName: "kube-api-access-7nxm8") pod "007ae1ce-fc4f-4ef9-944b-432df2c76108" (UID: "007ae1ce-fc4f-4ef9-944b-432df2c76108"). InnerVolumeSpecName "kube-api-access-7nxm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:02:35.955475 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:35.955352    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/007ae1ce-fc4f-4ef9-944b-432df2c76108-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "007ae1ce-fc4f-4ef9-944b-432df2c76108" (UID: "007ae1ce-fc4f-4ef9-944b-432df2c76108"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:02:35.982820 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:35.982788    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:36.054616 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:36.054585    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4r87x\" (UniqueName: \"kubernetes.io/projected/88d5dfba-057c-4513-a558-a9e8385361b9-kube-api-access-4r87x\") pod \"88d5dfba-057c-4513-a558-a9e8385361b9\" (UID: \"88d5dfba-057c-4513-a558-a9e8385361b9\") "
Sep 11 12:02:36.054808 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:36.054692    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88d5dfba-057c-4513-a558-a9e8385361b9-plugins-config-volume\") pod \"88d5dfba-057c-4513-a558-a9e8385361b9\" (UID: \"88d5dfba-057c-4513-a558-a9e8385361b9\") "
Sep 11 12:02:36.054942 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:36.054930    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nxm8\" (UniqueName: \"kubernetes.io/projected/007ae1ce-fc4f-4ef9-944b-432df2c76108-kube-api-access-7nxm8\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:02:36.054979 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:36.054945    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/007ae1ce-fc4f-4ef9-944b-432df2c76108-metrics-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:02:36.054979 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:36.054954    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/007ae1ce-fc4f-4ef9-944b-432df2c76108-maas-api-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:02:36.055045 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:36.055018    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88d5dfba-057c-4513-a558-a9e8385361b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "88d5dfba-057c-4513-a558-a9e8385361b9" (UID: "88d5dfba-057c-4513-a558-a9e8385361b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:02:36.056909 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:36.056889    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88d5dfba-057c-4513-a558-a9e8385361b9-kube-api-access-4r87x" (OuterVolumeSpecName: "kube-api-access-4r87x") pod "88d5dfba-057c-4513-a558-a9e8385361b9" (UID: "88d5dfba-057c-4513-a558-a9e8385361b9"). InnerVolumeSpecName "kube-api-access-4r87x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:02:36.155985 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:36.155911    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88d5dfba-057c-4513-a558-a9e8385361b9-plugins-config-volume\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:02:36.155985 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:36.155938    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r87x\" (UniqueName: \"kubernetes.io/projected/88d5dfba-057c-4513-a558-a9e8385361b9-kube-api-access-4r87x\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:02:36.357511 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:36.357478    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:36.853682 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:36.853649    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"
Sep 11 12:02:36.853682 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:36.853667    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc" event={"ID":"007ae1ce-fc4f-4ef9-944b-432df2c76108","Type":"ContainerDied","Data":"0e2e8329221e1a3c3c05ec8c4fc613e7859bbd18e676f191534a9873f117b34e"}
Sep 11 12:02:36.854235 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:36.853728    2575 scope.go:117] "RemoveContainer" containerID="fdee89962679d1aa4948c7111e78375ae11ceff640b2e8cbaa5dd225d6343bae"
Sep 11 12:02:36.855182 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:36.855150    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b" event={"ID":"88d5dfba-057c-4513-a558-a9e8385361b9","Type":"ContainerDied","Data":"f51e336d32eabbaca9ad1644480c520176b5bba46ec8738e0b59be23d9c3a6da"}
Sep 11 12:02:36.855286 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:36.855194    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b"
Sep 11 12:02:36.863439 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:36.863424    2575 scope.go:117] "RemoveContainer" containerID="2b8adaec3587f0521a2d1ea07e25eda8b28ceb4f9cdd6688a9ff88e881999256"
Sep 11 12:02:36.873536 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:36.873507    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b"]
Sep 11 12:02:36.877902 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:36.877879    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b8e0f0-7fdc66588b-bpp9b"]
Sep 11 12:02:36.891271 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:36.891246    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"]
Sep 11 12:02:36.902172 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:36.902147    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b8e0f0-79b559d964-2wsfc"]
Sep 11 12:02:38.081479 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:38.081449    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="007ae1ce-fc4f-4ef9-944b-432df2c76108" path="/var/lib/kubelet/pods/007ae1ce-fc4f-4ef9-944b-432df2c76108/volumes"
Sep 11 12:02:38.081864 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:38.081816    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88d5dfba-057c-4513-a558-a9e8385361b9" path="/var/lib/kubelet/pods/88d5dfba-057c-4513-a558-a9e8385361b9/volumes"
Sep 11 12:02:42.613017 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:42.612929    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:42.688068 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:42.688024    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:42.919210 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:42.919121    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:02:46.673588 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:02:46.673546    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:02.144761 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.144721    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"]
Sep 11 12:03:02.145229 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.144977    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9" podUID="186d630a-edbc-4637-a975-3398feb4cef9" containerName="maas-api" containerID="cri-o://e83ae1d49a5e6095e5dc185cd12b4aa570a700f6a6e71b328f3f81c60ffbc268" gracePeriod=30
Sep 11 12:03:02.187713 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.187686    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:02.229960 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.229931    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4"]
Sep 11 12:03:02.230191 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.230162    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4" podUID="20b52758-5ee3-432b-a6ef-3cc175d5df4d" containerName="payload-pre-processing" containerID="cri-o://d73248329c9b63e91632ed45f810a55945d345bf36bdfd4d776e4d5a6a36e8f3" gracePeriod=30
Sep 11 12:03:02.320911 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.320879    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:02.425828 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.425609    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:02.484347 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.484320    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"
Sep 11 12:03:02.517217 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.517193    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4"
Sep 11 12:03:02.532110 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.532081    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:02.616380 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.616349    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/186d630a-edbc-4637-a975-3398feb4cef9-maas-api-tls\") pod \"186d630a-edbc-4637-a975-3398feb4cef9\" (UID: \"186d630a-edbc-4637-a975-3398feb4cef9\") "
Sep 11 12:03:02.616554 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.616391    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/186d630a-edbc-4637-a975-3398feb4cef9-metrics-certs\") pod \"186d630a-edbc-4637-a975-3398feb4cef9\" (UID: \"186d630a-edbc-4637-a975-3398feb4cef9\") "
Sep 11 12:03:02.616554 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.616410    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhwbv\" (UniqueName: \"kubernetes.io/projected/186d630a-edbc-4637-a975-3398feb4cef9-kube-api-access-dhwbv\") pod \"186d630a-edbc-4637-a975-3398feb4cef9\" (UID: \"186d630a-edbc-4637-a975-3398feb4cef9\") "
Sep 11 12:03:02.616554 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.616431    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20b52758-5ee3-432b-a6ef-3cc175d5df4d-plugins-config-volume\") pod \"20b52758-5ee3-432b-a6ef-3cc175d5df4d\" (UID: \"20b52758-5ee3-432b-a6ef-3cc175d5df4d\") "
Sep 11 12:03:02.616741 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.616627    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmpf9\" (UniqueName: \"kubernetes.io/projected/20b52758-5ee3-432b-a6ef-3cc175d5df4d-kube-api-access-xmpf9\") pod \"20b52758-5ee3-432b-a6ef-3cc175d5df4d\" (UID: \"20b52758-5ee3-432b-a6ef-3cc175d5df4d\") "
Sep 11 12:03:02.616799 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.616754    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/186d630a-edbc-4637-a975-3398feb4cef9-openshift-service-ca\") pod \"186d630a-edbc-4637-a975-3398feb4cef9\" (UID: \"186d630a-edbc-4637-a975-3398feb4cef9\") "
Sep 11 12:03:02.616856 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.616817    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20b52758-5ee3-432b-a6ef-3cc175d5df4d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "20b52758-5ee3-432b-a6ef-3cc175d5df4d" (UID: "20b52758-5ee3-432b-a6ef-3cc175d5df4d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:03:02.617103 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.617082    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/186d630a-edbc-4637-a975-3398feb4cef9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "186d630a-edbc-4637-a975-3398feb4cef9" (UID: "186d630a-edbc-4637-a975-3398feb4cef9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:03:02.617202 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.617103    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20b52758-5ee3-432b-a6ef-3cc175d5df4d-plugins-config-volume\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:03:02.619271 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.619246    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186d630a-edbc-4637-a975-3398feb4cef9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "186d630a-edbc-4637-a975-3398feb4cef9" (UID: "186d630a-edbc-4637-a975-3398feb4cef9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:03:02.619376 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.619284    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186d630a-edbc-4637-a975-3398feb4cef9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "186d630a-edbc-4637-a975-3398feb4cef9" (UID: "186d630a-edbc-4637-a975-3398feb4cef9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:03:02.619376 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.619365    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/186d630a-edbc-4637-a975-3398feb4cef9-kube-api-access-dhwbv" (OuterVolumeSpecName: "kube-api-access-dhwbv") pod "186d630a-edbc-4637-a975-3398feb4cef9" (UID: "186d630a-edbc-4637-a975-3398feb4cef9"). InnerVolumeSpecName "kube-api-access-dhwbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:03:02.619476 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.619391    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20b52758-5ee3-432b-a6ef-3cc175d5df4d-kube-api-access-xmpf9" (OuterVolumeSpecName: "kube-api-access-xmpf9") pod "20b52758-5ee3-432b-a6ef-3cc175d5df4d" (UID: "20b52758-5ee3-432b-a6ef-3cc175d5df4d"). InnerVolumeSpecName "kube-api-access-xmpf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:03:02.718682 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.718586    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/186d630a-edbc-4637-a975-3398feb4cef9-maas-api-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:03:02.718682 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.718609    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/186d630a-edbc-4637-a975-3398feb4cef9-metrics-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:03:02.718682 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.718618    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhwbv\" (UniqueName: \"kubernetes.io/projected/186d630a-edbc-4637-a975-3398feb4cef9-kube-api-access-dhwbv\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:03:02.718682 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.718628    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmpf9\" (UniqueName: \"kubernetes.io/projected/20b52758-5ee3-432b-a6ef-3cc175d5df4d-kube-api-access-xmpf9\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:03:02.718682 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.718658    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/186d630a-edbc-4637-a975-3398feb4cef9-openshift-service-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:03:02.842543 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.842503    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:02.997247 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.997219    2575 generic.go:356] "Generic (PLEG): container finished" podID="186d630a-edbc-4637-a975-3398feb4cef9" containerID="e83ae1d49a5e6095e5dc185cd12b4aa570a700f6a6e71b328f3f81c60ffbc268" exitCode=0
Sep 11 12:03:02.997393 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.997266    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"
Sep 11 12:03:02.997393 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.997296    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9" event={"ID":"186d630a-edbc-4637-a975-3398feb4cef9","Type":"ContainerDied","Data":"e83ae1d49a5e6095e5dc185cd12b4aa570a700f6a6e71b328f3f81c60ffbc268"}
Sep 11 12:03:02.997393 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.997352    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9" event={"ID":"186d630a-edbc-4637-a975-3398feb4cef9","Type":"ContainerDied","Data":"741b56861df14ba008ae8f43bd23ee850f3c1e2e29e1da7d7193873cc9556ff0"}
Sep 11 12:03:02.997393 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.997378    2575 scope.go:117] "RemoveContainer" containerID="e83ae1d49a5e6095e5dc185cd12b4aa570a700f6a6e71b328f3f81c60ffbc268"
Sep 11 12:03:02.998838 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.998817    2575 generic.go:356] "Generic (PLEG): container finished" podID="20b52758-5ee3-432b-a6ef-3cc175d5df4d" containerID="d73248329c9b63e91632ed45f810a55945d345bf36bdfd4d776e4d5a6a36e8f3" exitCode=0
Sep 11 12:03:02.998966 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.998860    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4"
Sep 11 12:03:02.998966 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.998903    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4" event={"ID":"20b52758-5ee3-432b-a6ef-3cc175d5df4d","Type":"ContainerDied","Data":"d73248329c9b63e91632ed45f810a55945d345bf36bdfd4d776e4d5a6a36e8f3"}
Sep 11 12:03:02.998966 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:02.998953    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4" event={"ID":"20b52758-5ee3-432b-a6ef-3cc175d5df4d","Type":"ContainerDied","Data":"e419978fe60104d56a94fcdc4161d7c7cb4d64e5da8dcb4bb1ba359f5e595bde"}
Sep 11 12:03:03.007543 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:03.007527    2575 scope.go:117] "RemoveContainer" containerID="e83ae1d49a5e6095e5dc185cd12b4aa570a700f6a6e71b328f3f81c60ffbc268"
Sep 11 12:03:03.007948 ip-10-0-135-16 kubenswrapper[2575]: E0911 12:03:03.007921    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e83ae1d49a5e6095e5dc185cd12b4aa570a700f6a6e71b328f3f81c60ffbc268\": container with ID starting with e83ae1d49a5e6095e5dc185cd12b4aa570a700f6a6e71b328f3f81c60ffbc268 not found: ID does not exist" containerID="e83ae1d49a5e6095e5dc185cd12b4aa570a700f6a6e71b328f3f81c60ffbc268"
Sep 11 12:03:03.008045 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:03.007953    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e83ae1d49a5e6095e5dc185cd12b4aa570a700f6a6e71b328f3f81c60ffbc268"} err="failed to get container status \"e83ae1d49a5e6095e5dc185cd12b4aa570a700f6a6e71b328f3f81c60ffbc268\": rpc error: code = NotFound desc = could not find container \"e83ae1d49a5e6095e5dc185cd12b4aa570a700f6a6e71b328f3f81c60ffbc268\": container with ID starting with e83ae1d49a5e6095e5dc185cd12b4aa570a700f6a6e71b328f3f81c60ffbc268 not found: ID does not exist"
Sep 11 12:03:03.008045 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:03.007969    2575 scope.go:117] "RemoveContainer" containerID="d73248329c9b63e91632ed45f810a55945d345bf36bdfd4d776e4d5a6a36e8f3"
Sep 11 12:03:03.016415 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:03.016395    2575 scope.go:117] "RemoveContainer" containerID="d73248329c9b63e91632ed45f810a55945d345bf36bdfd4d776e4d5a6a36e8f3"
Sep 11 12:03:03.016694 ip-10-0-135-16 kubenswrapper[2575]: E0911 12:03:03.016674    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d73248329c9b63e91632ed45f810a55945d345bf36bdfd4d776e4d5a6a36e8f3\": container with ID starting with d73248329c9b63e91632ed45f810a55945d345bf36bdfd4d776e4d5a6a36e8f3 not found: ID does not exist" containerID="d73248329c9b63e91632ed45f810a55945d345bf36bdfd4d776e4d5a6a36e8f3"
Sep 11 12:03:03.016777 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:03.016699    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d73248329c9b63e91632ed45f810a55945d345bf36bdfd4d776e4d5a6a36e8f3"} err="failed to get container status \"d73248329c9b63e91632ed45f810a55945d345bf36bdfd4d776e4d5a6a36e8f3\": rpc error: code = NotFound desc = could not find container \"d73248329c9b63e91632ed45f810a55945d345bf36bdfd4d776e4d5a6a36e8f3\": container with ID starting with d73248329c9b63e91632ed45f810a55945d345bf36bdfd4d776e4d5a6a36e8f3 not found: ID does not exist"
Sep 11 12:03:03.025150 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:03.025130    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"]
Sep 11 12:03:03.030286 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:03.030262    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b19164-6c99699cdc-fp5w9"]
Sep 11 12:03:03.043015 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:03.042989    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4"]
Sep 11 12:03:03.053039 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:03.053015    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-b19164-78784585f8-nz6k4"]
Sep 11 12:03:04.081904 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:04.081871    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="186d630a-edbc-4637-a975-3398feb4cef9" path="/var/lib/kubelet/pods/186d630a-edbc-4637-a975-3398feb4cef9/volumes"
Sep 11 12:03:04.082270 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:04.082236    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20b52758-5ee3-432b-a6ef-3cc175d5df4d" path="/var/lib/kubelet/pods/20b52758-5ee3-432b-a6ef-3cc175d5df4d/volumes"
Sep 11 12:03:07.916399 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:07.916366    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:31.450035 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:31.450003    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:45.202941 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:45.202905    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:45.382565 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:45.382531    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:51.951537 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:51.951497    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:52.130208 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:52.130179    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:03:52.232534 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:03:52.232445    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:02.817380 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:02.817340    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:03.725528 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:03.725483    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:03.794379 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:03.794339    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x"]
Sep 11 12:04:03.794795 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:03.794604    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x" podUID="2b384fe0-a67d-4ce5-b58d-5277bd7b5770" containerName="payload-processing" containerID="cri-o://12d2ca9f2b437cd6f7f2467e213012de38229fb31217ebf1a8ba414aa5a534f1" gracePeriod=30
Sep 11 12:04:03.797467 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:03.797412    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl"]
Sep 11 12:04:03.798257 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:03.798181    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl" podUID="d7eb0d7c-f90e-42e5-a142-9a640b32683b" containerName="payload-pre-processing" containerID="cri-o://995eccf69b507f3bb0856091568856067b8fb98c5b2f96723ab18d543dad1a81" gracePeriod=30
Sep 11 12:04:03.859529 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:03.859502    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:04.045255 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.045218    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:04.079789 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.079761    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x"
Sep 11 12:04:04.082963 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.082942    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl"
Sep 11 12:04:04.213474 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.213444    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9d56\" (UniqueName: \"kubernetes.io/projected/d7eb0d7c-f90e-42e5-a142-9a640b32683b-kube-api-access-x9d56\") pod \"d7eb0d7c-f90e-42e5-a142-9a640b32683b\" (UID: \"d7eb0d7c-f90e-42e5-a142-9a640b32683b\") "
Sep 11 12:04:04.213474 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.213476    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qx2f8\" (UniqueName: \"kubernetes.io/projected/2b384fe0-a67d-4ce5-b58d-5277bd7b5770-kube-api-access-qx2f8\") pod \"2b384fe0-a67d-4ce5-b58d-5277bd7b5770\" (UID: \"2b384fe0-a67d-4ce5-b58d-5277bd7b5770\") "
Sep 11 12:04:04.213760 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.213521    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b384fe0-a67d-4ce5-b58d-5277bd7b5770-plugins-config-volume\") pod \"2b384fe0-a67d-4ce5-b58d-5277bd7b5770\" (UID: \"2b384fe0-a67d-4ce5-b58d-5277bd7b5770\") "
Sep 11 12:04:04.213760 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.213668    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7eb0d7c-f90e-42e5-a142-9a640b32683b-plugins-config-volume\") pod \"d7eb0d7c-f90e-42e5-a142-9a640b32683b\" (UID: \"d7eb0d7c-f90e-42e5-a142-9a640b32683b\") "
Sep 11 12:04:04.213928 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.213901    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b384fe0-a67d-4ce5-b58d-5277bd7b5770-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2b384fe0-a67d-4ce5-b58d-5277bd7b5770" (UID: "2b384fe0-a67d-4ce5-b58d-5277bd7b5770"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:04:04.214032 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.213995    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7eb0d7c-f90e-42e5-a142-9a640b32683b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d7eb0d7c-f90e-42e5-a142-9a640b32683b" (UID: "d7eb0d7c-f90e-42e5-a142-9a640b32683b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:04:04.214322 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.214086    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b384fe0-a67d-4ce5-b58d-5277bd7b5770-plugins-config-volume\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:04:04.214322 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.214107    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7eb0d7c-f90e-42e5-a142-9a640b32683b-plugins-config-volume\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:04:04.215682 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.215658    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b384fe0-a67d-4ce5-b58d-5277bd7b5770-kube-api-access-qx2f8" (OuterVolumeSpecName: "kube-api-access-qx2f8") pod "2b384fe0-a67d-4ce5-b58d-5277bd7b5770" (UID: "2b384fe0-a67d-4ce5-b58d-5277bd7b5770"). InnerVolumeSpecName "kube-api-access-qx2f8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:04:04.215682 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.215681    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7eb0d7c-f90e-42e5-a142-9a640b32683b-kube-api-access-x9d56" (OuterVolumeSpecName: "kube-api-access-x9d56") pod "d7eb0d7c-f90e-42e5-a142-9a640b32683b" (UID: "d7eb0d7c-f90e-42e5-a142-9a640b32683b"). InnerVolumeSpecName "kube-api-access-x9d56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:04:04.230106 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.230073    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:04.315524 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.315467    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9d56\" (UniqueName: \"kubernetes.io/projected/d7eb0d7c-f90e-42e5-a142-9a640b32683b-kube-api-access-x9d56\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:04:04.315524 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.315487    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qx2f8\" (UniqueName: \"kubernetes.io/projected/2b384fe0-a67d-4ce5-b58d-5277bd7b5770-kube-api-access-qx2f8\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:04:04.317668 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.317645    2575 generic.go:356] "Generic (PLEG): container finished" podID="d7eb0d7c-f90e-42e5-a142-9a640b32683b" containerID="995eccf69b507f3bb0856091568856067b8fb98c5b2f96723ab18d543dad1a81" exitCode=0
Sep 11 12:04:04.317772 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.317681    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl"
Sep 11 12:04:04.317772 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.317737    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl" event={"ID":"d7eb0d7c-f90e-42e5-a142-9a640b32683b","Type":"ContainerDied","Data":"995eccf69b507f3bb0856091568856067b8fb98c5b2f96723ab18d543dad1a81"}
Sep 11 12:04:04.317890 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.317780    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl" event={"ID":"d7eb0d7c-f90e-42e5-a142-9a640b32683b","Type":"ContainerDied","Data":"3ab8df467130804e63da9fef43a46c2edfea2f47b0cfd5be437e863b38a36413"}
Sep 11 12:04:04.317890 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.317802    2575 scope.go:117] "RemoveContainer" containerID="995eccf69b507f3bb0856091568856067b8fb98c5b2f96723ab18d543dad1a81"
Sep 11 12:04:04.319123 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.319103    2575 generic.go:356] "Generic (PLEG): container finished" podID="2b384fe0-a67d-4ce5-b58d-5277bd7b5770" containerID="12d2ca9f2b437cd6f7f2467e213012de38229fb31217ebf1a8ba414aa5a534f1" exitCode=0
Sep 11 12:04:04.319218 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.319134    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x" event={"ID":"2b384fe0-a67d-4ce5-b58d-5277bd7b5770","Type":"ContainerDied","Data":"12d2ca9f2b437cd6f7f2467e213012de38229fb31217ebf1a8ba414aa5a534f1"}
Sep 11 12:04:04.319218 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.319164    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x" event={"ID":"2b384fe0-a67d-4ce5-b58d-5277bd7b5770","Type":"ContainerDied","Data":"40143dad78f2199951ba9be40f37ad9a896ce4b7a784161ef24085d055af1577"}
Sep 11 12:04:04.319218 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.319212    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x"
Sep 11 12:04:04.327570 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.327551    2575 scope.go:117] "RemoveContainer" containerID="995eccf69b507f3bb0856091568856067b8fb98c5b2f96723ab18d543dad1a81"
Sep 11 12:04:04.327923 ip-10-0-135-16 kubenswrapper[2575]: E0911 12:04:04.327900    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"995eccf69b507f3bb0856091568856067b8fb98c5b2f96723ab18d543dad1a81\": container with ID starting with 995eccf69b507f3bb0856091568856067b8fb98c5b2f96723ab18d543dad1a81 not found: ID does not exist" containerID="995eccf69b507f3bb0856091568856067b8fb98c5b2f96723ab18d543dad1a81"
Sep 11 12:04:04.328003 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.327932    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"995eccf69b507f3bb0856091568856067b8fb98c5b2f96723ab18d543dad1a81"} err="failed to get container status \"995eccf69b507f3bb0856091568856067b8fb98c5b2f96723ab18d543dad1a81\": rpc error: code = NotFound desc = could not find container \"995eccf69b507f3bb0856091568856067b8fb98c5b2f96723ab18d543dad1a81\": container with ID starting with 995eccf69b507f3bb0856091568856067b8fb98c5b2f96723ab18d543dad1a81 not found: ID does not exist"
Sep 11 12:04:04.328003 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.327949    2575 scope.go:117] "RemoveContainer" containerID="12d2ca9f2b437cd6f7f2467e213012de38229fb31217ebf1a8ba414aa5a534f1"
Sep 11 12:04:04.335823 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.335808    2575 scope.go:117] "RemoveContainer" containerID="12d2ca9f2b437cd6f7f2467e213012de38229fb31217ebf1a8ba414aa5a534f1"
Sep 11 12:04:04.336078 ip-10-0-135-16 kubenswrapper[2575]: E0911 12:04:04.336056    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12d2ca9f2b437cd6f7f2467e213012de38229fb31217ebf1a8ba414aa5a534f1\": container with ID starting with 12d2ca9f2b437cd6f7f2467e213012de38229fb31217ebf1a8ba414aa5a534f1 not found: ID does not exist" containerID="12d2ca9f2b437cd6f7f2467e213012de38229fb31217ebf1a8ba414aa5a534f1"
Sep 11 12:04:04.336158 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.336081    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12d2ca9f2b437cd6f7f2467e213012de38229fb31217ebf1a8ba414aa5a534f1"} err="failed to get container status \"12d2ca9f2b437cd6f7f2467e213012de38229fb31217ebf1a8ba414aa5a534f1\": rpc error: code = NotFound desc = could not find container \"12d2ca9f2b437cd6f7f2467e213012de38229fb31217ebf1a8ba414aa5a534f1\": container with ID starting with 12d2ca9f2b437cd6f7f2467e213012de38229fb31217ebf1a8ba414aa5a534f1 not found: ID does not exist"
Sep 11 12:04:04.354121 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.354100    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl"]
Sep 11 12:04:04.358821 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.358802    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-720006-f77d45ffd-5chhl"]
Sep 11 12:04:04.375367 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.375339    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x"]
Sep 11 12:04:04.385086 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.385068    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-720006-5d4f946f89-zvm2x"]
Sep 11 12:04:04.528494 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:04.528463    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:06.082037 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:06.082005    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b384fe0-a67d-4ce5-b58d-5277bd7b5770" path="/var/lib/kubelet/pods/2b384fe0-a67d-4ce5-b58d-5277bd7b5770/volumes"
Sep 11 12:04:06.082395 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:06.082357    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7eb0d7c-f90e-42e5-a142-9a640b32683b" path="/var/lib/kubelet/pods/d7eb0d7c-f90e-42e5-a142-9a640b32683b/volumes"
Sep 11 12:04:11.215843 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:11.215804    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:11.317209 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:11.317178    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:11.587612 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:11.587580    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:12.014087 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:12.014048    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:22.740294 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:22.740258    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"]
Sep 11 12:04:22.740812 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:22.740487    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk" podUID="bb11417d-cbd4-47cb-9449-a2df504de3e1" containerName="maas-api" containerID="cri-o://4aab7b8af96ca1af2eeea4ed84833b8db6c5634d13a081a0b32953b8fc34c1cc" gracePeriod=30
Sep 11 12:04:22.805747 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:22.805712    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:22.855724 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:22.855682    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw"]
Sep 11 12:04:22.855953 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:22.855927    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw" podUID="5dc8042b-4520-4a85-b260-d3a7ba649f6b" containerName="payload-processing" containerID="cri-o://65e8507cb992d29b6f00a3ccc99ca637d5492aeb702d9189314ee596606446c0" gracePeriod=30
Sep 11 12:04:22.862202 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:22.862179    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn"]
Sep 11 12:04:22.862409 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:22.862368    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn" podUID="1b9fc270-59db-4dc0-9d41-b31866b9947f" containerName="payload-pre-processing" containerID="cri-o://7d690d1ae16ee0370eec1ba8ee529ac2218a38cbc71533e0b68fe2a8f24bd246" gracePeriod=30
Sep 11 12:04:22.940408 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:22.940359    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:23.008523 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.008491    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"
Sep 11 12:04:23.049866 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.049836    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:23.084786 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.084758    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb11417d-cbd4-47cb-9449-a2df504de3e1-metrics-certs\") pod \"bb11417d-cbd4-47cb-9449-a2df504de3e1\" (UID: \"bb11417d-cbd4-47cb-9449-a2df504de3e1\") "
Sep 11 12:04:23.084786 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.084795    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb11417d-cbd4-47cb-9449-a2df504de3e1-maas-api-tls\") pod \"bb11417d-cbd4-47cb-9449-a2df504de3e1\" (UID: \"bb11417d-cbd4-47cb-9449-a2df504de3e1\") "
Sep 11 12:04:23.085019 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.084840    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb11417d-cbd4-47cb-9449-a2df504de3e1-openshift-service-ca\") pod \"bb11417d-cbd4-47cb-9449-a2df504de3e1\" (UID: \"bb11417d-cbd4-47cb-9449-a2df504de3e1\") "
Sep 11 12:04:23.085019 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.084869    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vj2qj\" (UniqueName: \"kubernetes.io/projected/bb11417d-cbd4-47cb-9449-a2df504de3e1-kube-api-access-vj2qj\") pod \"bb11417d-cbd4-47cb-9449-a2df504de3e1\" (UID: \"bb11417d-cbd4-47cb-9449-a2df504de3e1\") "
Sep 11 12:04:23.085821 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.085710    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb11417d-cbd4-47cb-9449-a2df504de3e1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bb11417d-cbd4-47cb-9449-a2df504de3e1" (UID: "bb11417d-cbd4-47cb-9449-a2df504de3e1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:04:23.087801 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.087690    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb11417d-cbd4-47cb-9449-a2df504de3e1-kube-api-access-vj2qj" (OuterVolumeSpecName: "kube-api-access-vj2qj") pod "bb11417d-cbd4-47cb-9449-a2df504de3e1" (UID: "bb11417d-cbd4-47cb-9449-a2df504de3e1"). InnerVolumeSpecName "kube-api-access-vj2qj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:04:23.087801 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.087708    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb11417d-cbd4-47cb-9449-a2df504de3e1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bb11417d-cbd4-47cb-9449-a2df504de3e1" (UID: "bb11417d-cbd4-47cb-9449-a2df504de3e1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:04:23.089490 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.089468    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb11417d-cbd4-47cb-9449-a2df504de3e1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bb11417d-cbd4-47cb-9449-a2df504de3e1" (UID: "bb11417d-cbd4-47cb-9449-a2df504de3e1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:04:23.105938 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.105919    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw"
Sep 11 12:04:23.129389 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.129368    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn"
Sep 11 12:04:23.177533 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.177487    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:23.185558 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.185536    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqbzj\" (UniqueName: \"kubernetes.io/projected/5dc8042b-4520-4a85-b260-d3a7ba649f6b-kube-api-access-dqbzj\") pod \"5dc8042b-4520-4a85-b260-d3a7ba649f6b\" (UID: \"5dc8042b-4520-4a85-b260-d3a7ba649f6b\") "
Sep 11 12:04:23.185715 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.185576    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5dc8042b-4520-4a85-b260-d3a7ba649f6b-plugins-config-volume\") pod \"5dc8042b-4520-4a85-b260-d3a7ba649f6b\" (UID: \"5dc8042b-4520-4a85-b260-d3a7ba649f6b\") "
Sep 11 12:04:23.185715 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.185607    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b9fc270-59db-4dc0-9d41-b31866b9947f-plugins-config-volume\") pod \"1b9fc270-59db-4dc0-9d41-b31866b9947f\" (UID: \"1b9fc270-59db-4dc0-9d41-b31866b9947f\") "
Sep 11 12:04:23.185715 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.185656    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-756zz\" (UniqueName: \"kubernetes.io/projected/1b9fc270-59db-4dc0-9d41-b31866b9947f-kube-api-access-756zz\") pod \"1b9fc270-59db-4dc0-9d41-b31866b9947f\" (UID: \"1b9fc270-59db-4dc0-9d41-b31866b9947f\") "
Sep 11 12:04:23.185996 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.185976    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb11417d-cbd4-47cb-9449-a2df504de3e1-metrics-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:04:23.185996 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.185999    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb11417d-cbd4-47cb-9449-a2df504de3e1-maas-api-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:04:23.186163 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.186011    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb11417d-cbd4-47cb-9449-a2df504de3e1-openshift-service-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:04:23.186163 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.186024    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vj2qj\" (UniqueName: \"kubernetes.io/projected/bb11417d-cbd4-47cb-9449-a2df504de3e1-kube-api-access-vj2qj\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:04:23.186163 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.185972    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dc8042b-4520-4a85-b260-d3a7ba649f6b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5dc8042b-4520-4a85-b260-d3a7ba649f6b" (UID: "5dc8042b-4520-4a85-b260-d3a7ba649f6b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:04:23.186163 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.186007    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b9fc270-59db-4dc0-9d41-b31866b9947f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1b9fc270-59db-4dc0-9d41-b31866b9947f" (UID: "1b9fc270-59db-4dc0-9d41-b31866b9947f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:04:23.187936 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.187914    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b9fc270-59db-4dc0-9d41-b31866b9947f-kube-api-access-756zz" (OuterVolumeSpecName: "kube-api-access-756zz") pod "1b9fc270-59db-4dc0-9d41-b31866b9947f" (UID: "1b9fc270-59db-4dc0-9d41-b31866b9947f"). InnerVolumeSpecName "kube-api-access-756zz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:04:23.188012 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.187966    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dc8042b-4520-4a85-b260-d3a7ba649f6b-kube-api-access-dqbzj" (OuterVolumeSpecName: "kube-api-access-dqbzj") pod "5dc8042b-4520-4a85-b260-d3a7ba649f6b" (UID: "5dc8042b-4520-4a85-b260-d3a7ba649f6b"). InnerVolumeSpecName "kube-api-access-dqbzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:04:23.286947 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.286885    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqbzj\" (UniqueName: \"kubernetes.io/projected/5dc8042b-4520-4a85-b260-d3a7ba649f6b-kube-api-access-dqbzj\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:04:23.286947 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.286907    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5dc8042b-4520-4a85-b260-d3a7ba649f6b-plugins-config-volume\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:04:23.286947 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.286918    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b9fc270-59db-4dc0-9d41-b31866b9947f-plugins-config-volume\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:04:23.286947 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.286928    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-756zz\" (UniqueName: \"kubernetes.io/projected/1b9fc270-59db-4dc0-9d41-b31866b9947f-kube-api-access-756zz\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:04:23.418295 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.418264    2575 generic.go:356] "Generic (PLEG): container finished" podID="bb11417d-cbd4-47cb-9449-a2df504de3e1" containerID="4aab7b8af96ca1af2eeea4ed84833b8db6c5634d13a081a0b32953b8fc34c1cc" exitCode=0
Sep 11 12:04:23.418444 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.418306    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"
Sep 11 12:04:23.418444 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.418345    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk" event={"ID":"bb11417d-cbd4-47cb-9449-a2df504de3e1","Type":"ContainerDied","Data":"4aab7b8af96ca1af2eeea4ed84833b8db6c5634d13a081a0b32953b8fc34c1cc"}
Sep 11 12:04:23.418444 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.418393    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk" event={"ID":"bb11417d-cbd4-47cb-9449-a2df504de3e1","Type":"ContainerDied","Data":"aa2813e7f88f4ac4ca2934c5eaf4b36de80504da07f2c0cafce963d5b5651165"}
Sep 11 12:04:23.418444 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.418416    2575 scope.go:117] "RemoveContainer" containerID="4aab7b8af96ca1af2eeea4ed84833b8db6c5634d13a081a0b32953b8fc34c1cc"
Sep 11 12:04:23.419855 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.419821    2575 generic.go:356] "Generic (PLEG): container finished" podID="5dc8042b-4520-4a85-b260-d3a7ba649f6b" containerID="65e8507cb992d29b6f00a3ccc99ca637d5492aeb702d9189314ee596606446c0" exitCode=0
Sep 11 12:04:23.420131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.419856    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw"
Sep 11 12:04:23.420131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.419878    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw" event={"ID":"5dc8042b-4520-4a85-b260-d3a7ba649f6b","Type":"ContainerDied","Data":"65e8507cb992d29b6f00a3ccc99ca637d5492aeb702d9189314ee596606446c0"}
Sep 11 12:04:23.420131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.419927    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw" event={"ID":"5dc8042b-4520-4a85-b260-d3a7ba649f6b","Type":"ContainerDied","Data":"229279aca31a185d7b105e31a57d1f03b52c0ce242854b5d4ec69edcfca04d8e"}
Sep 11 12:04:23.421475 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.421453    2575 generic.go:356] "Generic (PLEG): container finished" podID="1b9fc270-59db-4dc0-9d41-b31866b9947f" containerID="7d690d1ae16ee0370eec1ba8ee529ac2218a38cbc71533e0b68fe2a8f24bd246" exitCode=0
Sep 11 12:04:23.421608 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.421485    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn"
Sep 11 12:04:23.421608 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.421528    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn" event={"ID":"1b9fc270-59db-4dc0-9d41-b31866b9947f","Type":"ContainerDied","Data":"7d690d1ae16ee0370eec1ba8ee529ac2218a38cbc71533e0b68fe2a8f24bd246"}
Sep 11 12:04:23.421608 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.421568    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn" event={"ID":"1b9fc270-59db-4dc0-9d41-b31866b9947f","Type":"ContainerDied","Data":"673834506bc64c157ba63f4f09cd2115cde5a00c5bdf59ea45f8b8308e19c58c"}
Sep 11 12:04:23.429051 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.429029    2575 scope.go:117] "RemoveContainer" containerID="4aab7b8af96ca1af2eeea4ed84833b8db6c5634d13a081a0b32953b8fc34c1cc"
Sep 11 12:04:23.429357 ip-10-0-135-16 kubenswrapper[2575]: E0911 12:04:23.429334    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4aab7b8af96ca1af2eeea4ed84833b8db6c5634d13a081a0b32953b8fc34c1cc\": container with ID starting with 4aab7b8af96ca1af2eeea4ed84833b8db6c5634d13a081a0b32953b8fc34c1cc not found: ID does not exist" containerID="4aab7b8af96ca1af2eeea4ed84833b8db6c5634d13a081a0b32953b8fc34c1cc"
Sep 11 12:04:23.429416 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.429369    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aab7b8af96ca1af2eeea4ed84833b8db6c5634d13a081a0b32953b8fc34c1cc"} err="failed to get container status \"4aab7b8af96ca1af2eeea4ed84833b8db6c5634d13a081a0b32953b8fc34c1cc\": rpc error: code = NotFound desc = could not find container \"4aab7b8af96ca1af2eeea4ed84833b8db6c5634d13a081a0b32953b8fc34c1cc\": container with ID starting with 4aab7b8af96ca1af2eeea4ed84833b8db6c5634d13a081a0b32953b8fc34c1cc not found: ID does not exist"
Sep 11 12:04:23.429416 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.429390    2575 scope.go:117] "RemoveContainer" containerID="65e8507cb992d29b6f00a3ccc99ca637d5492aeb702d9189314ee596606446c0"
Sep 11 12:04:23.443366 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.443349    2575 scope.go:117] "RemoveContainer" containerID="65e8507cb992d29b6f00a3ccc99ca637d5492aeb702d9189314ee596606446c0"
Sep 11 12:04:23.443691 ip-10-0-135-16 kubenswrapper[2575]: E0911 12:04:23.443670    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65e8507cb992d29b6f00a3ccc99ca637d5492aeb702d9189314ee596606446c0\": container with ID starting with 65e8507cb992d29b6f00a3ccc99ca637d5492aeb702d9189314ee596606446c0 not found: ID does not exist" containerID="65e8507cb992d29b6f00a3ccc99ca637d5492aeb702d9189314ee596606446c0"
Sep 11 12:04:23.443745 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.443699    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65e8507cb992d29b6f00a3ccc99ca637d5492aeb702d9189314ee596606446c0"} err="failed to get container status \"65e8507cb992d29b6f00a3ccc99ca637d5492aeb702d9189314ee596606446c0\": rpc error: code = NotFound desc = could not find container \"65e8507cb992d29b6f00a3ccc99ca637d5492aeb702d9189314ee596606446c0\": container with ID starting with 65e8507cb992d29b6f00a3ccc99ca637d5492aeb702d9189314ee596606446c0 not found: ID does not exist"
Sep 11 12:04:23.443745 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.443715    2575 scope.go:117] "RemoveContainer" containerID="7d690d1ae16ee0370eec1ba8ee529ac2218a38cbc71533e0b68fe2a8f24bd246"
Sep 11 12:04:23.449537 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.449518    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"]
Sep 11 12:04:23.452281 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.452261    2575 scope.go:117] "RemoveContainer" containerID="7d690d1ae16ee0370eec1ba8ee529ac2218a38cbc71533e0b68fe2a8f24bd246"
Sep 11 12:04:23.452618 ip-10-0-135-16 kubenswrapper[2575]: E0911 12:04:23.452596    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d690d1ae16ee0370eec1ba8ee529ac2218a38cbc71533e0b68fe2a8f24bd246\": container with ID starting with 7d690d1ae16ee0370eec1ba8ee529ac2218a38cbc71533e0b68fe2a8f24bd246 not found: ID does not exist" containerID="7d690d1ae16ee0370eec1ba8ee529ac2218a38cbc71533e0b68fe2a8f24bd246"
Sep 11 12:04:23.452702 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.452624    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d690d1ae16ee0370eec1ba8ee529ac2218a38cbc71533e0b68fe2a8f24bd246"} err="failed to get container status \"7d690d1ae16ee0370eec1ba8ee529ac2218a38cbc71533e0b68fe2a8f24bd246\": rpc error: code = NotFound desc = could not find container \"7d690d1ae16ee0370eec1ba8ee529ac2218a38cbc71533e0b68fe2a8f24bd246\": container with ID starting with 7d690d1ae16ee0370eec1ba8ee529ac2218a38cbc71533e0b68fe2a8f24bd246 not found: ID does not exist"
Sep 11 12:04:23.464252 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.464231    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-272c5e-7f8bf7fb67-cg8hk"]
Sep 11 12:04:23.487299 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.487276    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw"]
Sep 11 12:04:23.490286 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.490266    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-272c5e-6bc7b57547-zwktw"]
Sep 11 12:04:23.511461 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.511442    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn"]
Sep 11 12:04:23.515910 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.515890    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-272c5e-585d4b86c8-f8cvn"]
Sep 11 12:04:23.591718 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:23.591623    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:24.081675 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:24.081613    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b9fc270-59db-4dc0-9d41-b31866b9947f" path="/var/lib/kubelet/pods/1b9fc270-59db-4dc0-9d41-b31866b9947f/volumes"
Sep 11 12:04:24.082029 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:24.082017    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dc8042b-4520-4a85-b260-d3a7ba649f6b" path="/var/lib/kubelet/pods/5dc8042b-4520-4a85-b260-d3a7ba649f6b/volumes"
Sep 11 12:04:24.082360 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:24.082348    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb11417d-cbd4-47cb-9449-a2df504de3e1" path="/var/lib/kubelet/pods/bb11417d-cbd4-47cb-9449-a2df504de3e1/volumes"
Sep 11 12:04:30.656882 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:30.656845    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:30.790889 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:30.790843    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:30.956021 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:30.955932    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:31.435067 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:31.435033    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:42.019297 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:42.019264    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:42.121323 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:42.121296    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:42.238266 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:42.238233    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:42.330873 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:42.330794    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:42.675464 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:42.675367    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:50.472947 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:50.472907    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:53.228991 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:53.228955    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8bb47c4-fsxds"]
Sep 11 12:04:53.229388 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:53.229157    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-8bb47c4-fsxds" podUID="d061500f-6bec-4e87-965d-44533495bf4b" containerName="manager" containerID="cri-o://884e1dfe4ba27115df530eb56ce0e6d346b20f1b609c716409dd4aa075f6953d" gracePeriod=30
Sep 11 12:04:53.407467 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:53.407273    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:04:53.470154 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:53.470134    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8bb47c4-fsxds"
Sep 11 12:04:53.569640 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:53.569614    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d061500f-6bec-4e87-965d-44533495bf4b-webhook-cert\") pod \"d061500f-6bec-4e87-965d-44533495bf4b\" (UID: \"d061500f-6bec-4e87-965d-44533495bf4b\") "
Sep 11 12:04:53.569809 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:53.569675    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d061500f-6bec-4e87-965d-44533495bf4b-metrics-certs\") pod \"d061500f-6bec-4e87-965d-44533495bf4b\" (UID: \"d061500f-6bec-4e87-965d-44533495bf4b\") "
Sep 11 12:04:53.569809 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:53.569737    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfwf4\" (UniqueName: \"kubernetes.io/projected/d061500f-6bec-4e87-965d-44533495bf4b-kube-api-access-wfwf4\") pod \"d061500f-6bec-4e87-965d-44533495bf4b\" (UID: \"d061500f-6bec-4e87-965d-44533495bf4b\") "
Sep 11 12:04:53.571793 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:53.571757    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d061500f-6bec-4e87-965d-44533495bf4b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d061500f-6bec-4e87-965d-44533495bf4b" (UID: "d061500f-6bec-4e87-965d-44533495bf4b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:04:53.571936 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:53.571878    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d061500f-6bec-4e87-965d-44533495bf4b-kube-api-access-wfwf4" (OuterVolumeSpecName: "kube-api-access-wfwf4") pod "d061500f-6bec-4e87-965d-44533495bf4b" (UID: "d061500f-6bec-4e87-965d-44533495bf4b"). InnerVolumeSpecName "kube-api-access-wfwf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:04:53.571936 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:53.571886    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d061500f-6bec-4e87-965d-44533495bf4b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d061500f-6bec-4e87-965d-44533495bf4b" (UID: "d061500f-6bec-4e87-965d-44533495bf4b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:04:53.577605 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:53.577516    2575 generic.go:356] "Generic (PLEG): container finished" podID="d061500f-6bec-4e87-965d-44533495bf4b" containerID="884e1dfe4ba27115df530eb56ce0e6d346b20f1b609c716409dd4aa075f6953d" exitCode=0
Sep 11 12:04:53.577605 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:53.577568    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8bb47c4-fsxds"
Sep 11 12:04:53.577760 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:53.577600    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8bb47c4-fsxds" event={"ID":"d061500f-6bec-4e87-965d-44533495bf4b","Type":"ContainerDied","Data":"884e1dfe4ba27115df530eb56ce0e6d346b20f1b609c716409dd4aa075f6953d"}
Sep 11 12:04:53.577760 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:53.577677    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8bb47c4-fsxds" event={"ID":"d061500f-6bec-4e87-965d-44533495bf4b","Type":"ContainerDied","Data":"322a0488c9b2c5c2227754e0ef70ff49b31b70b30bd4f167559d7ad649088b4e"}
Sep 11 12:04:53.577760 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:53.577708    2575 scope.go:117] "RemoveContainer" containerID="884e1dfe4ba27115df530eb56ce0e6d346b20f1b609c716409dd4aa075f6953d"
Sep 11 12:04:53.590481 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:53.590466    2575 scope.go:117] "RemoveContainer" containerID="884e1dfe4ba27115df530eb56ce0e6d346b20f1b609c716409dd4aa075f6953d"
Sep 11 12:04:53.590782 ip-10-0-135-16 kubenswrapper[2575]: E0911 12:04:53.590760    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"884e1dfe4ba27115df530eb56ce0e6d346b20f1b609c716409dd4aa075f6953d\": container with ID starting with 884e1dfe4ba27115df530eb56ce0e6d346b20f1b609c716409dd4aa075f6953d not found: ID does not exist" containerID="884e1dfe4ba27115df530eb56ce0e6d346b20f1b609c716409dd4aa075f6953d"
Sep 11 12:04:53.590876 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:53.590783    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"884e1dfe4ba27115df530eb56ce0e6d346b20f1b609c716409dd4aa075f6953d"} err="failed to get container status \"884e1dfe4ba27115df530eb56ce0e6d346b20f1b609c716409dd4aa075f6953d\": rpc error: code = NotFound desc = could not find container \"884e1dfe4ba27115df530eb56ce0e6d346b20f1b609c716409dd4aa075f6953d\": container with ID starting with 884e1dfe4ba27115df530eb56ce0e6d346b20f1b609c716409dd4aa075f6953d not found: ID does not exist"
Sep 11 12:04:53.602380 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:53.602355    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8bb47c4-fsxds"]
Sep 11 12:04:53.608159 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:53.608139    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8bb47c4-fsxds"]
Sep 11 12:04:53.671034 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:53.671014    2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d061500f-6bec-4e87-965d-44533495bf4b-webhook-cert\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:04:53.671034 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:53.671034    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d061500f-6bec-4e87-965d-44533495bf4b-metrics-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:04:53.671151 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:53.671043    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfwf4\" (UniqueName: \"kubernetes.io/projected/d061500f-6bec-4e87-965d-44533495bf4b-kube-api-access-wfwf4\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:04:54.081189 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:04:54.081158    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d061500f-6bec-4e87-965d-44533495bf4b" path="/var/lib/kubelet/pods/d061500f-6bec-4e87-965d-44533495bf4b/volumes"
Sep 11 12:05:14.468649 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:14.468603    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:14.571303 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:14.571268    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:32.866248 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:32.866209    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:34.889963 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:34.889925    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:05:54.453485 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.453402    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8bb47c4-kvgqp"]
Sep 11 12:05:54.453874 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.453823    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20b52758-5ee3-432b-a6ef-3cc175d5df4d" containerName="payload-pre-processing"
Sep 11 12:05:54.453874 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.453837    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="20b52758-5ee3-432b-a6ef-3cc175d5df4d" containerName="payload-pre-processing"
Sep 11 12:05:54.453874 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.453847    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="007ae1ce-fc4f-4ef9-944b-432df2c76108" containerName="maas-api"
Sep 11 12:05:54.453874 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.453853    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="007ae1ce-fc4f-4ef9-944b-432df2c76108" containerName="maas-api"
Sep 11 12:05:54.453874 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.453864    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dc8042b-4520-4a85-b260-d3a7ba649f6b" containerName="payload-processing"
Sep 11 12:05:54.453874 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.453870    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dc8042b-4520-4a85-b260-d3a7ba649f6b" containerName="payload-processing"
Sep 11 12:05:54.453874 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.453877    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88d5dfba-057c-4513-a558-a9e8385361b9" containerName="payload-pre-processing"
Sep 11 12:05:54.454131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.453883    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="88d5dfba-057c-4513-a558-a9e8385361b9" containerName="payload-pre-processing"
Sep 11 12:05:54.454131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.453890    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d061500f-6bec-4e87-965d-44533495bf4b" containerName="manager"
Sep 11 12:05:54.454131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.453896    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d061500f-6bec-4e87-965d-44533495bf4b" containerName="manager"
Sep 11 12:05:54.454131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.453902    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="186d630a-edbc-4637-a975-3398feb4cef9" containerName="maas-api"
Sep 11 12:05:54.454131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.453907    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="186d630a-edbc-4637-a975-3398feb4cef9" containerName="maas-api"
Sep 11 12:05:54.454131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.453916    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b9fc270-59db-4dc0-9d41-b31866b9947f" containerName="payload-pre-processing"
Sep 11 12:05:54.454131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.453921    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b9fc270-59db-4dc0-9d41-b31866b9947f" containerName="payload-pre-processing"
Sep 11 12:05:54.454131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.453926    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b384fe0-a67d-4ce5-b58d-5277bd7b5770" containerName="payload-processing"
Sep 11 12:05:54.454131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.453932    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b384fe0-a67d-4ce5-b58d-5277bd7b5770" containerName="payload-processing"
Sep 11 12:05:54.454131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.453949    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7eb0d7c-f90e-42e5-a142-9a640b32683b" containerName="payload-pre-processing"
Sep 11 12:05:54.454131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.453954    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7eb0d7c-f90e-42e5-a142-9a640b32683b" containerName="payload-pre-processing"
Sep 11 12:05:54.454131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.453961    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb11417d-cbd4-47cb-9449-a2df504de3e1" containerName="maas-api"
Sep 11 12:05:54.454131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.453966    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb11417d-cbd4-47cb-9449-a2df504de3e1" containerName="maas-api"
Sep 11 12:05:54.454131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.454033    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="186d630a-edbc-4637-a975-3398feb4cef9" containerName="maas-api"
Sep 11 12:05:54.454131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.454041    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d061500f-6bec-4e87-965d-44533495bf4b" containerName="manager"
Sep 11 12:05:54.454131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.454047    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="20b52758-5ee3-432b-a6ef-3cc175d5df4d" containerName="payload-pre-processing"
Sep 11 12:05:54.454131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.454054    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b9fc270-59db-4dc0-9d41-b31866b9947f" containerName="payload-pre-processing"
Sep 11 12:05:54.454131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.454059    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7eb0d7c-f90e-42e5-a142-9a640b32683b" containerName="payload-pre-processing"
Sep 11 12:05:54.454131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.454065    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="88d5dfba-057c-4513-a558-a9e8385361b9" containerName="payload-pre-processing"
Sep 11 12:05:54.454131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.454071    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="007ae1ce-fc4f-4ef9-944b-432df2c76108" containerName="maas-api"
Sep 11 12:05:54.454131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.454078    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b384fe0-a67d-4ce5-b58d-5277bd7b5770" containerName="payload-processing"
Sep 11 12:05:54.454131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.454083    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dc8042b-4520-4a85-b260-d3a7ba649f6b" containerName="payload-processing"
Sep 11 12:05:54.454131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.454089    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb11417d-cbd4-47cb-9449-a2df504de3e1" containerName="maas-api"
Sep 11 12:05:54.457119 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.457094    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8bb47c4-kvgqp"
Sep 11 12:05:54.460471 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.460454    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9fdz2\""
Sep 11 12:05:54.461436 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.461412    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 12:05:54.461550 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.461475    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 12:05:54.461706 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.461689    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 12:05:54.469785 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.469760    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8bb47c4-kvgqp"]
Sep 11 12:05:54.508796 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.508769    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/575523e0-8a23-43d3-b8bd-31197b001581-metrics-certs\") pod \"maas-controller-8bb47c4-kvgqp\" (UID: \"575523e0-8a23-43d3-b8bd-31197b001581\") " pod="opendatahub/maas-controller-8bb47c4-kvgqp"
Sep 11 12:05:54.508954 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.508817    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/575523e0-8a23-43d3-b8bd-31197b001581-webhook-cert\") pod \"maas-controller-8bb47c4-kvgqp\" (UID: \"575523e0-8a23-43d3-b8bd-31197b001581\") " pod="opendatahub/maas-controller-8bb47c4-kvgqp"
Sep 11 12:05:54.508954 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.508865    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh9nk\" (UniqueName: \"kubernetes.io/projected/575523e0-8a23-43d3-b8bd-31197b001581-kube-api-access-rh9nk\") pod \"maas-controller-8bb47c4-kvgqp\" (UID: \"575523e0-8a23-43d3-b8bd-31197b001581\") " pod="opendatahub/maas-controller-8bb47c4-kvgqp"
Sep 11 12:05:54.609213 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.609182    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh9nk\" (UniqueName: \"kubernetes.io/projected/575523e0-8a23-43d3-b8bd-31197b001581-kube-api-access-rh9nk\") pod \"maas-controller-8bb47c4-kvgqp\" (UID: \"575523e0-8a23-43d3-b8bd-31197b001581\") " pod="opendatahub/maas-controller-8bb47c4-kvgqp"
Sep 11 12:05:54.609407 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.609274    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/575523e0-8a23-43d3-b8bd-31197b001581-metrics-certs\") pod \"maas-controller-8bb47c4-kvgqp\" (UID: \"575523e0-8a23-43d3-b8bd-31197b001581\") " pod="opendatahub/maas-controller-8bb47c4-kvgqp"
Sep 11 12:05:54.609407 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.609331    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/575523e0-8a23-43d3-b8bd-31197b001581-webhook-cert\") pod \"maas-controller-8bb47c4-kvgqp\" (UID: \"575523e0-8a23-43d3-b8bd-31197b001581\") " pod="opendatahub/maas-controller-8bb47c4-kvgqp"
Sep 11 12:05:54.611885 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.611861    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/575523e0-8a23-43d3-b8bd-31197b001581-metrics-certs\") pod \"maas-controller-8bb47c4-kvgqp\" (UID: \"575523e0-8a23-43d3-b8bd-31197b001581\") " pod="opendatahub/maas-controller-8bb47c4-kvgqp"
Sep 11 12:05:54.612005 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.611938    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/575523e0-8a23-43d3-b8bd-31197b001581-webhook-cert\") pod \"maas-controller-8bb47c4-kvgqp\" (UID: \"575523e0-8a23-43d3-b8bd-31197b001581\") " pod="opendatahub/maas-controller-8bb47c4-kvgqp"
Sep 11 12:05:54.618698 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.618680    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh9nk\" (UniqueName: \"kubernetes.io/projected/575523e0-8a23-43d3-b8bd-31197b001581-kube-api-access-rh9nk\") pod \"maas-controller-8bb47c4-kvgqp\" (UID: \"575523e0-8a23-43d3-b8bd-31197b001581\") " pod="opendatahub/maas-controller-8bb47c4-kvgqp"
Sep 11 12:05:54.768820 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:54.768777    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8bb47c4-kvgqp"
Sep 11 12:05:55.047056 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:55.047033    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8bb47c4-kvgqp"]
Sep 11 12:05:55.049259 ip-10-0-135-16 kubenswrapper[2575]: W0911 12:05:55.049229    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod575523e0_8a23_43d3_b8bd_31197b001581.slice/crio-abb94ad483905f258c817a5479af633f9ae95a1841a22f071a32042b94bd18aa WatchSource:0}: Error finding container abb94ad483905f258c817a5479af633f9ae95a1841a22f071a32042b94bd18aa: Status 404 returned error can't find the container with id abb94ad483905f258c817a5479af633f9ae95a1841a22f071a32042b94bd18aa
Sep 11 12:05:55.050492 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:55.050473    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 12:05:55.884999 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:55.884976    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8bb47c4-kvgqp" event={"ID":"575523e0-8a23-43d3-b8bd-31197b001581","Type":"ContainerStarted","Data":"abb94ad483905f258c817a5479af633f9ae95a1841a22f071a32042b94bd18aa"}
Sep 11 12:05:56.891507 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:56.891474    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8bb47c4-kvgqp" event={"ID":"575523e0-8a23-43d3-b8bd-31197b001581","Type":"ContainerStarted","Data":"34ab57321c5ed6665755efedffb89b31460e0c6a25bf10c3079e0f643c073ebd"}
Sep 11 12:05:56.891926 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:56.891537    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8bb47c4-kvgqp"
Sep 11 12:05:56.918277 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:05:56.918231    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8bb47c4-kvgqp" podStartSLOduration=2.091561548 podStartE2EDuration="2.918214721s" totalImagesPullingTime="826.653173ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:05:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:05:56.917299986 +0000 UTC m=+1897.337906015" watchObservedRunningTime="2026-09-11 12:05:56.918214721 +0000 UTC m=+1897.338820747"
Sep 11 12:06:07.898922 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:07.898881    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8bb47c4-kvgqp"
Sep 11 12:06:15.190941 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:15.190902    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:15.297198 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:15.297162    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8bb47c4-kvgqp"]
Sep 11 12:06:15.297445 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:15.297401    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-8bb47c4-kvgqp" podUID="575523e0-8a23-43d3-b8bd-31197b001581" containerName="manager" containerID="cri-o://34ab57321c5ed6665755efedffb89b31460e0c6a25bf10c3079e0f643c073ebd" gracePeriod=30
Sep 11 12:06:15.541833 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:15.541808    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8bb47c4-kvgqp"
Sep 11 12:06:15.711443 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:15.711409    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rh9nk\" (UniqueName: \"kubernetes.io/projected/575523e0-8a23-43d3-b8bd-31197b001581-kube-api-access-rh9nk\") pod \"575523e0-8a23-43d3-b8bd-31197b001581\" (UID: \"575523e0-8a23-43d3-b8bd-31197b001581\") "
Sep 11 12:06:15.711619 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:15.711492    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/575523e0-8a23-43d3-b8bd-31197b001581-webhook-cert\") pod \"575523e0-8a23-43d3-b8bd-31197b001581\" (UID: \"575523e0-8a23-43d3-b8bd-31197b001581\") "
Sep 11 12:06:15.711619 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:15.711557    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/575523e0-8a23-43d3-b8bd-31197b001581-metrics-certs\") pod \"575523e0-8a23-43d3-b8bd-31197b001581\" (UID: \"575523e0-8a23-43d3-b8bd-31197b001581\") "
Sep 11 12:06:15.713982 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:15.713936    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/575523e0-8a23-43d3-b8bd-31197b001581-kube-api-access-rh9nk" (OuterVolumeSpecName: "kube-api-access-rh9nk") pod "575523e0-8a23-43d3-b8bd-31197b001581" (UID: "575523e0-8a23-43d3-b8bd-31197b001581"). InnerVolumeSpecName "kube-api-access-rh9nk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:06:15.713982 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:15.713959    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/575523e0-8a23-43d3-b8bd-31197b001581-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "575523e0-8a23-43d3-b8bd-31197b001581" (UID: "575523e0-8a23-43d3-b8bd-31197b001581"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:06:15.713982 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:15.713948    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/575523e0-8a23-43d3-b8bd-31197b001581-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "575523e0-8a23-43d3-b8bd-31197b001581" (UID: "575523e0-8a23-43d3-b8bd-31197b001581"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:06:15.812693 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:15.812623    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rh9nk\" (UniqueName: \"kubernetes.io/projected/575523e0-8a23-43d3-b8bd-31197b001581-kube-api-access-rh9nk\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:06:15.812693 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:15.812690    2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/575523e0-8a23-43d3-b8bd-31197b001581-webhook-cert\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:06:15.812693 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:15.812700    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/575523e0-8a23-43d3-b8bd-31197b001581-metrics-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:06:15.987452 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:15.987416    2575 generic.go:356] "Generic (PLEG): container finished" podID="575523e0-8a23-43d3-b8bd-31197b001581" containerID="34ab57321c5ed6665755efedffb89b31460e0c6a25bf10c3079e0f643c073ebd" exitCode=0
Sep 11 12:06:15.987608 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:15.987463    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8bb47c4-kvgqp"
Sep 11 12:06:15.987608 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:15.987499    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8bb47c4-kvgqp" event={"ID":"575523e0-8a23-43d3-b8bd-31197b001581","Type":"ContainerDied","Data":"34ab57321c5ed6665755efedffb89b31460e0c6a25bf10c3079e0f643c073ebd"}
Sep 11 12:06:15.987608 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:15.987555    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8bb47c4-kvgqp" event={"ID":"575523e0-8a23-43d3-b8bd-31197b001581","Type":"ContainerDied","Data":"abb94ad483905f258c817a5479af633f9ae95a1841a22f071a32042b94bd18aa"}
Sep 11 12:06:15.987608 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:15.987584    2575 scope.go:117] "RemoveContainer" containerID="34ab57321c5ed6665755efedffb89b31460e0c6a25bf10c3079e0f643c073ebd"
Sep 11 12:06:15.997409 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:15.997392    2575 scope.go:117] "RemoveContainer" containerID="34ab57321c5ed6665755efedffb89b31460e0c6a25bf10c3079e0f643c073ebd"
Sep 11 12:06:15.997691 ip-10-0-135-16 kubenswrapper[2575]: E0911 12:06:15.997673    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34ab57321c5ed6665755efedffb89b31460e0c6a25bf10c3079e0f643c073ebd\": container with ID starting with 34ab57321c5ed6665755efedffb89b31460e0c6a25bf10c3079e0f643c073ebd not found: ID does not exist" containerID="34ab57321c5ed6665755efedffb89b31460e0c6a25bf10c3079e0f643c073ebd"
Sep 11 12:06:15.997761 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:15.997699    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34ab57321c5ed6665755efedffb89b31460e0c6a25bf10c3079e0f643c073ebd"} err="failed to get container status \"34ab57321c5ed6665755efedffb89b31460e0c6a25bf10c3079e0f643c073ebd\": rpc error: code = NotFound desc = could not find container \"34ab57321c5ed6665755efedffb89b31460e0c6a25bf10c3079e0f643c073ebd\": container with ID starting with 34ab57321c5ed6665755efedffb89b31460e0c6a25bf10c3079e0f643c073ebd not found: ID does not exist"
Sep 11 12:06:16.013571 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:16.013551    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8bb47c4-kvgqp"]
Sep 11 12:06:16.019123 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:16.019102    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8bb47c4-kvgqp"]
Sep 11 12:06:16.082137 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:16.082112    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="575523e0-8a23-43d3-b8bd-31197b001581" path="/var/lib/kubelet/pods/575523e0-8a23-43d3-b8bd-31197b001581/volumes"
Sep 11 12:06:16.960783 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:16.960752    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:06:17.070150 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:17.070117    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8bb47c4-j95vn"]
Sep 11 12:06:17.070510 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:17.070498    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="575523e0-8a23-43d3-b8bd-31197b001581" containerName="manager"
Sep 11 12:06:17.070566 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:17.070511    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="575523e0-8a23-43d3-b8bd-31197b001581" containerName="manager"
Sep 11 12:06:17.070605 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:17.070597    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="575523e0-8a23-43d3-b8bd-31197b001581" containerName="manager"
Sep 11 12:06:17.075207 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:17.075181    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8bb47c4-j95vn"
Sep 11 12:06:17.079462 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:17.079433    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 12:06:17.079602 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:17.079513    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 12:06:17.079602 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:17.079435    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9fdz2\""
Sep 11 12:06:17.079602 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:17.079436    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 12:06:17.096129 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:17.096100    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8bb47c4-j95vn"]
Sep 11 12:06:17.225555 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:17.225465    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1493da64-9aac-4abd-a5ef-c26036b2bdc6-metrics-certs\") pod \"maas-controller-8bb47c4-j95vn\" (UID: \"1493da64-9aac-4abd-a5ef-c26036b2bdc6\") " pod="opendatahub/maas-controller-8bb47c4-j95vn"
Sep 11 12:06:17.225555 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:17.225511    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46wjz\" (UniqueName: \"kubernetes.io/projected/1493da64-9aac-4abd-a5ef-c26036b2bdc6-kube-api-access-46wjz\") pod \"maas-controller-8bb47c4-j95vn\" (UID: \"1493da64-9aac-4abd-a5ef-c26036b2bdc6\") " pod="opendatahub/maas-controller-8bb47c4-j95vn"
Sep 11 12:06:17.225770 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:17.225660    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1493da64-9aac-4abd-a5ef-c26036b2bdc6-webhook-cert\") pod \"maas-controller-8bb47c4-j95vn\" (UID: \"1493da64-9aac-4abd-a5ef-c26036b2bdc6\") " pod="opendatahub/maas-controller-8bb47c4-j95vn"
Sep 11 12:06:17.326464 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:17.326428    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1493da64-9aac-4abd-a5ef-c26036b2bdc6-metrics-certs\") pod \"maas-controller-8bb47c4-j95vn\" (UID: \"1493da64-9aac-4abd-a5ef-c26036b2bdc6\") " pod="opendatahub/maas-controller-8bb47c4-j95vn"
Sep 11 12:06:17.326676 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:17.326478    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46wjz\" (UniqueName: \"kubernetes.io/projected/1493da64-9aac-4abd-a5ef-c26036b2bdc6-kube-api-access-46wjz\") pod \"maas-controller-8bb47c4-j95vn\" (UID: \"1493da64-9aac-4abd-a5ef-c26036b2bdc6\") " pod="opendatahub/maas-controller-8bb47c4-j95vn"
Sep 11 12:06:17.326676 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:17.326566    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1493da64-9aac-4abd-a5ef-c26036b2bdc6-webhook-cert\") pod \"maas-controller-8bb47c4-j95vn\" (UID: \"1493da64-9aac-4abd-a5ef-c26036b2bdc6\") " pod="opendatahub/maas-controller-8bb47c4-j95vn"
Sep 11 12:06:17.329114 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:17.329088    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1493da64-9aac-4abd-a5ef-c26036b2bdc6-webhook-cert\") pod \"maas-controller-8bb47c4-j95vn\" (UID: \"1493da64-9aac-4abd-a5ef-c26036b2bdc6\") " pod="opendatahub/maas-controller-8bb47c4-j95vn"
Sep 11 12:06:17.329220 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:17.329168    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1493da64-9aac-4abd-a5ef-c26036b2bdc6-metrics-certs\") pod \"maas-controller-8bb47c4-j95vn\" (UID: \"1493da64-9aac-4abd-a5ef-c26036b2bdc6\") " pod="opendatahub/maas-controller-8bb47c4-j95vn"
Sep 11 12:06:17.336608 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:17.336586    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46wjz\" (UniqueName: \"kubernetes.io/projected/1493da64-9aac-4abd-a5ef-c26036b2bdc6-kube-api-access-46wjz\") pod \"maas-controller-8bb47c4-j95vn\" (UID: \"1493da64-9aac-4abd-a5ef-c26036b2bdc6\") " pod="opendatahub/maas-controller-8bb47c4-j95vn"
Sep 11 12:06:17.386825 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:17.386801    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8bb47c4-j95vn"
Sep 11 12:06:17.669699 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:17.669656    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8bb47c4-j95vn"]
Sep 11 12:06:17.672340 ip-10-0-135-16 kubenswrapper[2575]: W0911 12:06:17.672303    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1493da64_9aac_4abd_a5ef_c26036b2bdc6.slice/crio-3c0d423a3e216791fe93ca89b79a6776a55f78bbc6fca80518100034e7989101 WatchSource:0}: Error finding container 3c0d423a3e216791fe93ca89b79a6776a55f78bbc6fca80518100034e7989101: Status 404 returned error can't find the container with id 3c0d423a3e216791fe93ca89b79a6776a55f78bbc6fca80518100034e7989101
Sep 11 12:06:17.999790 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:17.999750    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8bb47c4-j95vn" event={"ID":"1493da64-9aac-4abd-a5ef-c26036b2bdc6","Type":"ContainerStarted","Data":"3c0d423a3e216791fe93ca89b79a6776a55f78bbc6fca80518100034e7989101"}
Sep 11 12:06:19.006386 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:19.006352    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8bb47c4-j95vn" event={"ID":"1493da64-9aac-4abd-a5ef-c26036b2bdc6","Type":"ContainerStarted","Data":"80963c19bdfc1e5822f46bfcb8bf72c604f2c913243a24325717510935af2314"}
Sep 11 12:06:19.006854 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:19.006454    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8bb47c4-j95vn"
Sep 11 12:06:19.031164 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:19.031117    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8bb47c4-j95vn" podStartSLOduration=1.5315000479999998 podStartE2EDuration="2.031101036s" totalImagesPullingTime="499.600988ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:06:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:06:19.02759846 +0000 UTC m=+1919.448204486" watchObservedRunningTime="2026-09-11 12:06:19.031101036 +0000 UTC m=+1919.451707063"
Sep 11 12:06:30.014592 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:06:30.014562    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8bb47c4-j95vn"
Sep 11 12:08:58.515673 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.515586    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"]
Sep 11 12:08:58.519421 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.519396    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:08:58.522619 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.522593    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-58fdae53-kserve-self-signed-certs\""
Sep 11 12:08:58.542681 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.542656    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"]
Sep 11 12:08:58.641950 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.641917    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-tmp-dir\") pod \"e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") " pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:08:58.642098 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.641964    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-dshm\") pod \"e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") " pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:08:58.642098 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.641985    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-home\") pod \"e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") " pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:08:58.642098 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.642009    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-tls-certs\") pod \"e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") " pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:08:58.642098 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.642047    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kvv2\" (UniqueName: \"kubernetes.io/projected/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-kube-api-access-4kvv2\") pod \"e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") " pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:08:58.642098 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.642068    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-model-cache\") pod \"e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") " pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:08:58.668380 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.668355    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:08:58.742438 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.742409    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-tmp-dir\") pod \"e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") " pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:08:58.742579 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.742448    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-dshm\") pod \"e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") " pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:08:58.742579 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.742468    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-home\") pod \"e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") " pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:08:58.742579 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.742483    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-tls-certs\") pod \"e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") " pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:08:58.742579 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.742505    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kvv2\" (UniqueName: \"kubernetes.io/projected/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-kube-api-access-4kvv2\") pod \"e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") " pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:08:58.742579 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.742542    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-model-cache\") pod \"e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") " pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:08:58.742931 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.742910    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-tmp-dir\") pod \"e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") " pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:08:58.743011 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.742959    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-home\") pod \"e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") " pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:08:58.743011 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.742994    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-model-cache\") pod \"e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") " pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:08:58.744915 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.744896    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-dshm\") pod \"e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") " pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:08:58.745164 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.745145    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-tls-certs\") pod \"e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") " pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:08:58.759997 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.759968    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kvv2\" (UniqueName: \"kubernetes.io/projected/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-kube-api-access-4kvv2\") pod \"e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") " pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:08:58.828514 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:58.828458    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:08:59.101516 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:59.101491    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"]
Sep 11 12:08:59.103062 ip-10-0-135-16 kubenswrapper[2575]: W0911 12:08:59.103035    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0af76c75_b6bb_4a22_a3c7_1cd14a57a72d.slice/crio-b5ec29e5ef6aa1b010076d16c079283d944b634e234f5aa9312f0423fd682d96 WatchSource:0}: Error finding container b5ec29e5ef6aa1b010076d16c079283d944b634e234f5aa9312f0423fd682d96: Status 404 returned error can't find the container with id b5ec29e5ef6aa1b010076d16c079283d944b634e234f5aa9312f0423fd682d96
Sep 11 12:08:59.792583 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:59.792552    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm" event={"ID":"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d","Type":"ContainerStarted","Data":"2f5812e88920596f6dd3bd30227f3d65e5cade3cd1749df655666cf7f7692f5e"}
Sep 11 12:08:59.792976 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:59.792590    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm" event={"ID":"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d","Type":"ContainerStarted","Data":"b5ec29e5ef6aa1b010076d16c079283d944b634e234f5aa9312f0423fd682d96"}
Sep 11 12:08:59.813268 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:08:59.813229    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm" podStartSLOduration=1.813215354 podStartE2EDuration="1.813215354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:08:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:08:59.812455139 +0000 UTC m=+2080.233061168" watchObservedRunningTime="2026-09-11 12:08:59.813215354 +0000 UTC m=+2080.233821381"
Sep 11 12:09:08.828740 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:08.828703    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:09:08.828740 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:08.828748    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:09:08.842392 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:08.842367    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:09:08.857503 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:08.857475    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:09:10.831847 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:10.831812    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"]
Sep 11 12:09:10.835291 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:10.835268    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:10.838215 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:10.838191    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-58fdae53-kserve-self-signed-certs\""
Sep 11 12:09:10.855352 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:10.855332    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"]
Sep 11 12:09:10.920102 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:10.920073    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:10.945506 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:10.945474    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-home\") pod \"e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") " pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:10.945695 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:10.945537    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-dshm\") pod \"e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") " pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:10.945695 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:10.945563    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-tmp-dir\") pod \"e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") " pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:10.945695 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:10.945619    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-model-cache\") pod \"e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") " pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:10.945827 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:10.945746    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d094686-9e50-4773-8b63-b8243c8822ff-tls-certs\") pod \"e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") " pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:10.945865 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:10.945828    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmhc5\" (UniqueName: \"kubernetes.io/projected/6d094686-9e50-4773-8b63-b8243c8822ff-kube-api-access-wmhc5\") pod \"e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") " pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:11.047271 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:11.047236    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-home\") pod \"e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") " pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:11.047443 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:11.047302    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-dshm\") pod \"e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") " pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:11.047443 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:11.047322    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-tmp-dir\") pod \"e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") " pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:11.047443 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:11.047355    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-model-cache\") pod \"e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") " pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:11.047443 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:11.047395    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d094686-9e50-4773-8b63-b8243c8822ff-tls-certs\") pod \"e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") " pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:11.047719 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:11.047458    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmhc5\" (UniqueName: \"kubernetes.io/projected/6d094686-9e50-4773-8b63-b8243c8822ff-kube-api-access-wmhc5\") pod \"e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") " pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:11.047785 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:11.047723    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-home\") pod \"e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") " pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:11.047875 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:11.047856    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-tmp-dir\") pod \"e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") " pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:11.047929 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:11.047903    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-model-cache\") pod \"e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") " pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:11.049928 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:11.049910    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-dshm\") pod \"e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") " pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:11.050167 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:11.050150    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d094686-9e50-4773-8b63-b8243c8822ff-tls-certs\") pod \"e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") " pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:11.055812 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:11.055786    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmhc5\" (UniqueName: \"kubernetes.io/projected/6d094686-9e50-4773-8b63-b8243c8822ff-kube-api-access-wmhc5\") pod \"e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") " pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:11.145676 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:11.145614    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:11.420837 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:11.420812    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"]
Sep 11 12:09:11.422465 ip-10-0-135-16 kubenswrapper[2575]: W0911 12:09:11.422443    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d094686_9e50_4773_8b63_b8243c8822ff.slice/crio-2a4cce60ee4f13ce89d04cdfa6b5857160f3829691a9107d079629e9b69e72a9 WatchSource:0}: Error finding container 2a4cce60ee4f13ce89d04cdfa6b5857160f3829691a9107d079629e9b69e72a9: Status 404 returned error can't find the container with id 2a4cce60ee4f13ce89d04cdfa6b5857160f3829691a9107d079629e9b69e72a9
Sep 11 12:09:11.853558 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:11.853520    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt" event={"ID":"6d094686-9e50-4773-8b63-b8243c8822ff","Type":"ContainerStarted","Data":"01fbfbfc0707a3085e7f7dd630937c28f4eb853dec473d6e8758992863b332b8"}
Sep 11 12:09:11.853975 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:11.853572    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt" event={"ID":"6d094686-9e50-4773-8b63-b8243c8822ff","Type":"ContainerStarted","Data":"2a4cce60ee4f13ce89d04cdfa6b5857160f3829691a9107d079629e9b69e72a9"}
Sep 11 12:09:21.146309 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:21.146268    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:21.146309 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:21.146315    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:21.159255 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:21.159233    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:21.194071 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:21.194021    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt" podStartSLOduration=11.194008437 podStartE2EDuration="11.194008437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:09:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:09:11.879330199 +0000 UTC m=+2092.299936249" watchObservedRunningTime="2026-09-11 12:09:21.194008437 +0000 UTC m=+2101.614614463"
Sep 11 12:09:21.913950 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:21.913922    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:31.806544 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:31.806504    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"]
Sep 11 12:09:31.831624 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:31.831599    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:33.508122 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:33.508090    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:33.748367 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:33.748335    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"]
Sep 11 12:09:33.775695 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:33.775611    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:09:46.825951 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:46.825890    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm" podUID="0af76c75-b6bb-4a22-a3c7-1cd14a57a72d" containerName="main" containerID="cri-o://2f5812e88920596f6dd3bd30227f3d65e5cade3cd1749df655666cf7f7692f5e" gracePeriod=45
Sep 11 12:09:47.032198 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:47.032169    2575 generic.go:356] "Generic (PLEG): container finished" podID="0af76c75-b6bb-4a22-a3c7-1cd14a57a72d" containerID="2f5812e88920596f6dd3bd30227f3d65e5cade3cd1749df655666cf7f7692f5e" exitCode=0
Sep 11 12:09:47.032335 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:47.032242    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm" event={"ID":"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d","Type":"ContainerDied","Data":"2f5812e88920596f6dd3bd30227f3d65e5cade3cd1749df655666cf7f7692f5e"}
Sep 11 12:09:47.078089 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:47.078035    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:09:47.179908 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:47.179878    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kvv2\" (UniqueName: \"kubernetes.io/projected/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-kube-api-access-4kvv2\") pod \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") "
Sep 11 12:09:47.180064 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:47.179931    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-home\") pod \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") "
Sep 11 12:09:47.180064 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:47.179984    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-tmp-dir\") pod \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") "
Sep 11 12:09:47.180064 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:47.180003    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-model-cache\") pod \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") "
Sep 11 12:09:47.180064 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:47.180047    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-dshm\") pod \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") "
Sep 11 12:09:47.180274 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:47.180087    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-tls-certs\") pod \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\" (UID: \"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d\") "
Sep 11 12:09:47.180274 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:47.180219    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0af76c75-b6bb-4a22-a3c7-1cd14a57a72d" (UID: "0af76c75-b6bb-4a22-a3c7-1cd14a57a72d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:09:47.180379 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:47.180290    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-home" (OuterVolumeSpecName: "home") pod "0af76c75-b6bb-4a22-a3c7-1cd14a57a72d" (UID: "0af76c75-b6bb-4a22-a3c7-1cd14a57a72d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:09:47.180557 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:47.180442    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-home\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:09:47.180557 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:47.180464    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-tmp-dir\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:09:47.180557 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:47.180516    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-model-cache" (OuterVolumeSpecName: "model-cache") pod "0af76c75-b6bb-4a22-a3c7-1cd14a57a72d" (UID: "0af76c75-b6bb-4a22-a3c7-1cd14a57a72d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:09:47.182239 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:47.182212    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-dshm" (OuterVolumeSpecName: "dshm") pod "0af76c75-b6bb-4a22-a3c7-1cd14a57a72d" (UID: "0af76c75-b6bb-4a22-a3c7-1cd14a57a72d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:09:47.182347 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:47.182257    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-kube-api-access-4kvv2" (OuterVolumeSpecName: "kube-api-access-4kvv2") pod "0af76c75-b6bb-4a22-a3c7-1cd14a57a72d" (UID: "0af76c75-b6bb-4a22-a3c7-1cd14a57a72d"). InnerVolumeSpecName "kube-api-access-4kvv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:09:47.182408 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:47.182380    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0af76c75-b6bb-4a22-a3c7-1cd14a57a72d" (UID: "0af76c75-b6bb-4a22-a3c7-1cd14a57a72d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:09:47.281316 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:47.281290    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kvv2\" (UniqueName: \"kubernetes.io/projected/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-kube-api-access-4kvv2\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:09:47.281316 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:47.281315    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-model-cache\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:09:47.281477 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:47.281325    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-dshm\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:09:47.281477 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:47.281332    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d-tls-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:09:48.039549 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:48.039524    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"
Sep 11 12:09:48.039979 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:48.039557    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm" event={"ID":"0af76c75-b6bb-4a22-a3c7-1cd14a57a72d","Type":"ContainerDied","Data":"b5ec29e5ef6aa1b010076d16c079283d944b634e234f5aa9312f0423fd682d96"}
Sep 11 12:09:48.039979 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:48.039605    2575 scope.go:117] "RemoveContainer" containerID="2f5812e88920596f6dd3bd30227f3d65e5cade3cd1749df655666cf7f7692f5e"
Sep 11 12:09:48.062168 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:48.062145    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"]
Sep 11 12:09:48.067310 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:48.067288    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-58fdae53-kserve-5bf76ff587-gh4gm"]
Sep 11 12:09:48.081196 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:48.081171    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0af76c75-b6bb-4a22-a3c7-1cd14a57a72d" path="/var/lib/kubelet/pods/0af76c75-b6bb-4a22-a3c7-1cd14a57a72d/volumes"
Sep 11 12:09:48.767703 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:48.767666    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt" podUID="6d094686-9e50-4773-8b63-b8243c8822ff" containerName="main" containerID="cri-o://01fbfbfc0707a3085e7f7dd630937c28f4eb853dec473d6e8758992863b332b8" gracePeriod=45
Sep 11 12:09:49.018353 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.018302    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:49.051177 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.051128    2575 generic.go:356] "Generic (PLEG): container finished" podID="6d094686-9e50-4773-8b63-b8243c8822ff" containerID="01fbfbfc0707a3085e7f7dd630937c28f4eb853dec473d6e8758992863b332b8" exitCode=0
Sep 11 12:09:49.051177 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.051174    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"
Sep 11 12:09:49.051671 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.051180    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt" event={"ID":"6d094686-9e50-4773-8b63-b8243c8822ff","Type":"ContainerDied","Data":"01fbfbfc0707a3085e7f7dd630937c28f4eb853dec473d6e8758992863b332b8"}
Sep 11 12:09:49.051671 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.051221    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt" event={"ID":"6d094686-9e50-4773-8b63-b8243c8822ff","Type":"ContainerDied","Data":"2a4cce60ee4f13ce89d04cdfa6b5857160f3829691a9107d079629e9b69e72a9"}
Sep 11 12:09:49.051671 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.051248    2575 scope.go:117] "RemoveContainer" containerID="01fbfbfc0707a3085e7f7dd630937c28f4eb853dec473d6e8758992863b332b8"
Sep 11 12:09:49.059846 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.059827    2575 scope.go:117] "RemoveContainer" containerID="01fbfbfc0707a3085e7f7dd630937c28f4eb853dec473d6e8758992863b332b8"
Sep 11 12:09:49.060087 ip-10-0-135-16 kubenswrapper[2575]: E0911 12:09:49.060069    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01fbfbfc0707a3085e7f7dd630937c28f4eb853dec473d6e8758992863b332b8\": container with ID starting with 01fbfbfc0707a3085e7f7dd630937c28f4eb853dec473d6e8758992863b332b8 not found: ID does not exist" containerID="01fbfbfc0707a3085e7f7dd630937c28f4eb853dec473d6e8758992863b332b8"
Sep 11 12:09:49.060154 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.060103    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01fbfbfc0707a3085e7f7dd630937c28f4eb853dec473d6e8758992863b332b8"} err="failed to get container status \"01fbfbfc0707a3085e7f7dd630937c28f4eb853dec473d6e8758992863b332b8\": rpc error: code = NotFound desc = could not find container \"01fbfbfc0707a3085e7f7dd630937c28f4eb853dec473d6e8758992863b332b8\": container with ID starting with 01fbfbfc0707a3085e7f7dd630937c28f4eb853dec473d6e8758992863b332b8 not found: ID does not exist"
Sep 11 12:09:49.099782 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.099761    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-dshm\") pod \"6d094686-9e50-4773-8b63-b8243c8822ff\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") "
Sep 11 12:09:49.099878 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.099797    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmhc5\" (UniqueName: \"kubernetes.io/projected/6d094686-9e50-4773-8b63-b8243c8822ff-kube-api-access-wmhc5\") pod \"6d094686-9e50-4773-8b63-b8243c8822ff\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") "
Sep 11 12:09:49.099878 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.099831    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d094686-9e50-4773-8b63-b8243c8822ff-tls-certs\") pod \"6d094686-9e50-4773-8b63-b8243c8822ff\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") "
Sep 11 12:09:49.099878 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.099873    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-tmp-dir\") pod \"6d094686-9e50-4773-8b63-b8243c8822ff\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") "
Sep 11 12:09:49.100035 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.099893    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-home\") pod \"6d094686-9e50-4773-8b63-b8243c8822ff\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") "
Sep 11 12:09:49.100035 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.099947    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-model-cache\") pod \"6d094686-9e50-4773-8b63-b8243c8822ff\" (UID: \"6d094686-9e50-4773-8b63-b8243c8822ff\") "
Sep 11 12:09:49.100201 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.100154    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-home" (OuterVolumeSpecName: "home") pod "6d094686-9e50-4773-8b63-b8243c8822ff" (UID: "6d094686-9e50-4773-8b63-b8243c8822ff"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:09:49.100259 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.100197    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6d094686-9e50-4773-8b63-b8243c8822ff" (UID: "6d094686-9e50-4773-8b63-b8243c8822ff"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:09:49.100341 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.100322    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-tmp-dir\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:09:49.100396 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.100346    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-home\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:09:49.100396 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.100338    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-model-cache" (OuterVolumeSpecName: "model-cache") pod "6d094686-9e50-4773-8b63-b8243c8822ff" (UID: "6d094686-9e50-4773-8b63-b8243c8822ff"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:09:49.102050 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.102029    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d094686-9e50-4773-8b63-b8243c8822ff-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6d094686-9e50-4773-8b63-b8243c8822ff" (UID: "6d094686-9e50-4773-8b63-b8243c8822ff"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:09:49.102513 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.102487    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-dshm" (OuterVolumeSpecName: "dshm") pod "6d094686-9e50-4773-8b63-b8243c8822ff" (UID: "6d094686-9e50-4773-8b63-b8243c8822ff"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 12:09:49.102513 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.102499    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d094686-9e50-4773-8b63-b8243c8822ff-kube-api-access-wmhc5" (OuterVolumeSpecName: "kube-api-access-wmhc5") pod "6d094686-9e50-4773-8b63-b8243c8822ff" (UID: "6d094686-9e50-4773-8b63-b8243c8822ff"). InnerVolumeSpecName "kube-api-access-wmhc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:09:49.201373 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.201348    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-dshm\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:09:49.201373 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.201373    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmhc5\" (UniqueName: \"kubernetes.io/projected/6d094686-9e50-4773-8b63-b8243c8822ff-kube-api-access-wmhc5\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:09:49.201561 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.201384    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d094686-9e50-4773-8b63-b8243c8822ff-tls-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:09:49.201561 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.201393    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d094686-9e50-4773-8b63-b8243c8822ff-model-cache\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:09:49.373808 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.373762    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"]
Sep 11 12:09:49.377845 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:49.377820    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-58fdae53-kserve-6bdb97f695-7kbkt"]
Sep 11 12:09:50.081846 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:50.081815    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d094686-9e50-4773-8b63-b8243c8822ff" path="/var/lib/kubelet/pods/6d094686-9e50-4773-8b63-b8243c8822ff/volumes"
Sep 11 12:09:53.851395 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:53.851367    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-796ffbb698-gdpz8"]
Sep 11 12:09:53.853797 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:53.851759    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d094686-9e50-4773-8b63-b8243c8822ff" containerName="main"
Sep 11 12:09:53.853797 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:53.851771    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d094686-9e50-4773-8b63-b8243c8822ff" containerName="main"
Sep 11 12:09:53.853797 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:53.851788    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0af76c75-b6bb-4a22-a3c7-1cd14a57a72d" containerName="main"
Sep 11 12:09:53.853797 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:53.851794    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0af76c75-b6bb-4a22-a3c7-1cd14a57a72d" containerName="main"
Sep 11 12:09:53.853797 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:53.851877    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d094686-9e50-4773-8b63-b8243c8822ff" containerName="main"
Sep 11 12:09:53.853797 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:53.851886    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0af76c75-b6bb-4a22-a3c7-1cd14a57a72d" containerName="main"
Sep 11 12:09:53.854738 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:53.854723    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-796ffbb698-gdpz8"
Sep 11 12:09:53.870765 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:53.870738    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-796ffbb698-gdpz8"]
Sep 11 12:09:53.943855 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:53.943820    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9d049be-d977-449c-ad4c-26954a781551-maas-api-tls\") pod \"maas-api-796ffbb698-gdpz8\" (UID: \"b9d049be-d977-449c-ad4c-26954a781551\") " pod="odh-ai-gateway-infra/maas-api-796ffbb698-gdpz8"
Sep 11 12:09:53.944027 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:53.943886    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mmsk\" (UniqueName: \"kubernetes.io/projected/b9d049be-d977-449c-ad4c-26954a781551-kube-api-access-9mmsk\") pod \"maas-api-796ffbb698-gdpz8\" (UID: \"b9d049be-d977-449c-ad4c-26954a781551\") " pod="odh-ai-gateway-infra/maas-api-796ffbb698-gdpz8"
Sep 11 12:09:53.944027 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:53.943989    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9d049be-d977-449c-ad4c-26954a781551-metrics-certs\") pod \"maas-api-796ffbb698-gdpz8\" (UID: \"b9d049be-d977-449c-ad4c-26954a781551\") " pod="odh-ai-gateway-infra/maas-api-796ffbb698-gdpz8"
Sep 11 12:09:53.944123 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:53.944027    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9d049be-d977-449c-ad4c-26954a781551-openshift-service-ca\") pod \"maas-api-796ffbb698-gdpz8\" (UID: \"b9d049be-d977-449c-ad4c-26954a781551\") " pod="odh-ai-gateway-infra/maas-api-796ffbb698-gdpz8"
Sep 11 12:09:54.044672 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:54.044628    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9d049be-d977-449c-ad4c-26954a781551-metrics-certs\") pod \"maas-api-796ffbb698-gdpz8\" (UID: \"b9d049be-d977-449c-ad4c-26954a781551\") " pod="odh-ai-gateway-infra/maas-api-796ffbb698-gdpz8"
Sep 11 12:09:54.044672 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:54.044673    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9d049be-d977-449c-ad4c-26954a781551-openshift-service-ca\") pod \"maas-api-796ffbb698-gdpz8\" (UID: \"b9d049be-d977-449c-ad4c-26954a781551\") " pod="odh-ai-gateway-infra/maas-api-796ffbb698-gdpz8"
Sep 11 12:09:54.044896 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:54.044696    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9d049be-d977-449c-ad4c-26954a781551-maas-api-tls\") pod \"maas-api-796ffbb698-gdpz8\" (UID: \"b9d049be-d977-449c-ad4c-26954a781551\") " pod="odh-ai-gateway-infra/maas-api-796ffbb698-gdpz8"
Sep 11 12:09:54.044896 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:54.044734    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mmsk\" (UniqueName: \"kubernetes.io/projected/b9d049be-d977-449c-ad4c-26954a781551-kube-api-access-9mmsk\") pod \"maas-api-796ffbb698-gdpz8\" (UID: \"b9d049be-d977-449c-ad4c-26954a781551\") " pod="odh-ai-gateway-infra/maas-api-796ffbb698-gdpz8"
Sep 11 12:09:54.045341 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:54.045317    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9d049be-d977-449c-ad4c-26954a781551-openshift-service-ca\") pod \"maas-api-796ffbb698-gdpz8\" (UID: \"b9d049be-d977-449c-ad4c-26954a781551\") " pod="odh-ai-gateway-infra/maas-api-796ffbb698-gdpz8"
Sep 11 12:09:54.047112 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:54.047091    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9d049be-d977-449c-ad4c-26954a781551-maas-api-tls\") pod \"maas-api-796ffbb698-gdpz8\" (UID: \"b9d049be-d977-449c-ad4c-26954a781551\") " pod="odh-ai-gateway-infra/maas-api-796ffbb698-gdpz8"
Sep 11 12:09:54.047197 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:54.047145    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9d049be-d977-449c-ad4c-26954a781551-metrics-certs\") pod \"maas-api-796ffbb698-gdpz8\" (UID: \"b9d049be-d977-449c-ad4c-26954a781551\") " pod="odh-ai-gateway-infra/maas-api-796ffbb698-gdpz8"
Sep 11 12:09:54.053312 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:54.053292    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mmsk\" (UniqueName: \"kubernetes.io/projected/b9d049be-d977-449c-ad4c-26954a781551-kube-api-access-9mmsk\") pod \"maas-api-796ffbb698-gdpz8\" (UID: \"b9d049be-d977-449c-ad4c-26954a781551\") " pod="odh-ai-gateway-infra/maas-api-796ffbb698-gdpz8"
Sep 11 12:09:54.165341 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:54.165262    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-796ffbb698-gdpz8"
Sep 11 12:09:54.443543 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:54.443520    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-796ffbb698-gdpz8"]
Sep 11 12:09:54.446359 ip-10-0-135-16 kubenswrapper[2575]: W0911 12:09:54.446335    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9d049be_d977_449c_ad4c_26954a781551.slice/crio-b839683904df586ee14e6dd48795326a6aeb9d461703f3237f8ff2bb4c34bb68 WatchSource:0}: Error finding container b839683904df586ee14e6dd48795326a6aeb9d461703f3237f8ff2bb4c34bb68: Status 404 returned error can't find the container with id b839683904df586ee14e6dd48795326a6aeb9d461703f3237f8ff2bb4c34bb68
Sep 11 12:09:55.083451 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:55.083422    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-796ffbb698-gdpz8" event={"ID":"b9d049be-d977-449c-ad4c-26954a781551","Type":"ContainerStarted","Data":"16b903b02b2a797d70c667473d3bb059ae7472363b6c2a257576ac61742c4dd6"}
Sep 11 12:09:55.083823 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:55.083467    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-796ffbb698-gdpz8" event={"ID":"b9d049be-d977-449c-ad4c-26954a781551","Type":"ContainerStarted","Data":"b839683904df586ee14e6dd48795326a6aeb9d461703f3237f8ff2bb4c34bb68"}
Sep 11 12:09:55.083823 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:55.083559    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-796ffbb698-gdpz8"
Sep 11 12:09:55.105426 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:09:55.105384    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-796ffbb698-gdpz8" podStartSLOduration=1.65772341 podStartE2EDuration="2.105370955s" totalImagesPullingTime="447.647545ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:09:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:09:55.105019097 +0000 UTC m=+2135.525625130" watchObservedRunningTime="2026-09-11 12:09:55.105370955 +0000 UTC m=+2135.525976982"
Sep 11 12:10:01.093271 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:01.093238    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-796ffbb698-gdpz8"
Sep 11 12:10:10.429054 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:10.429016    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"]
Sep 11 12:10:10.429586 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:10.429217    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79" podUID="4060759e-15b8-4435-aa5b-9ffd03644ede" containerName="maas-api" containerID="cri-o://e73d47aacef629f907185253673fb8174ab1238500883e4a2d2bd98f812386c2" gracePeriod=30
Sep 11 12:10:11.689129 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:11.689105    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"
Sep 11 12:10:11.809566 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:11.809487    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4060759e-15b8-4435-aa5b-9ffd03644ede-maas-api-tls\") pod \"4060759e-15b8-4435-aa5b-9ffd03644ede\" (UID: \"4060759e-15b8-4435-aa5b-9ffd03644ede\") "
Sep 11 12:10:11.809743 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:11.809579    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45p6h\" (UniqueName: \"kubernetes.io/projected/4060759e-15b8-4435-aa5b-9ffd03644ede-kube-api-access-45p6h\") pod \"4060759e-15b8-4435-aa5b-9ffd03644ede\" (UID: \"4060759e-15b8-4435-aa5b-9ffd03644ede\") "
Sep 11 12:10:11.809743 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:11.809660    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4060759e-15b8-4435-aa5b-9ffd03644ede-openshift-service-ca\") pod \"4060759e-15b8-4435-aa5b-9ffd03644ede\" (UID: \"4060759e-15b8-4435-aa5b-9ffd03644ede\") "
Sep 11 12:10:11.809743 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:11.809715    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4060759e-15b8-4435-aa5b-9ffd03644ede-metrics-certs\") pod \"4060759e-15b8-4435-aa5b-9ffd03644ede\" (UID: \"4060759e-15b8-4435-aa5b-9ffd03644ede\") "
Sep 11 12:10:11.810075 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:11.810044    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4060759e-15b8-4435-aa5b-9ffd03644ede-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4060759e-15b8-4435-aa5b-9ffd03644ede" (UID: "4060759e-15b8-4435-aa5b-9ffd03644ede"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 12:10:11.811885 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:11.811862    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4060759e-15b8-4435-aa5b-9ffd03644ede-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4060759e-15b8-4435-aa5b-9ffd03644ede" (UID: "4060759e-15b8-4435-aa5b-9ffd03644ede"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:10:11.811992 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:11.811880    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4060759e-15b8-4435-aa5b-9ffd03644ede-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4060759e-15b8-4435-aa5b-9ffd03644ede" (UID: "4060759e-15b8-4435-aa5b-9ffd03644ede"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 12:10:11.811992 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:11.811966    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4060759e-15b8-4435-aa5b-9ffd03644ede-kube-api-access-45p6h" (OuterVolumeSpecName: "kube-api-access-45p6h") pod "4060759e-15b8-4435-aa5b-9ffd03644ede" (UID: "4060759e-15b8-4435-aa5b-9ffd03644ede"). InnerVolumeSpecName "kube-api-access-45p6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 12:10:11.911220 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:11.911192    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4060759e-15b8-4435-aa5b-9ffd03644ede-maas-api-tls\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:10:11.911220 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:11.911219    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45p6h\" (UniqueName: \"kubernetes.io/projected/4060759e-15b8-4435-aa5b-9ffd03644ede-kube-api-access-45p6h\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:10:11.911412 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:11.911235    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4060759e-15b8-4435-aa5b-9ffd03644ede-openshift-service-ca\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:10:11.911412 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:11.911246    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4060759e-15b8-4435-aa5b-9ffd03644ede-metrics-certs\") on node \"ip-10-0-135-16.ec2.internal\" DevicePath \"\""
Sep 11 12:10:12.166102 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:12.166067    2575 generic.go:356] "Generic (PLEG): container finished" podID="4060759e-15b8-4435-aa5b-9ffd03644ede" containerID="e73d47aacef629f907185253673fb8174ab1238500883e4a2d2bd98f812386c2" exitCode=0
Sep 11 12:10:12.166267 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:12.166116    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"
Sep 11 12:10:12.166267 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:12.166153    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79" event={"ID":"4060759e-15b8-4435-aa5b-9ffd03644ede","Type":"ContainerDied","Data":"e73d47aacef629f907185253673fb8174ab1238500883e4a2d2bd98f812386c2"}
Sep 11 12:10:12.166267 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:12.166214    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79" event={"ID":"4060759e-15b8-4435-aa5b-9ffd03644ede","Type":"ContainerDied","Data":"79120349297d8105b2369b606b40f096dc51b235d4c6f0b935f48aa34c311f11"}
Sep 11 12:10:12.166267 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:12.166241    2575 scope.go:117] "RemoveContainer" containerID="e73d47aacef629f907185253673fb8174ab1238500883e4a2d2bd98f812386c2"
Sep 11 12:10:12.175416 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:12.175398    2575 scope.go:117] "RemoveContainer" containerID="e73d47aacef629f907185253673fb8174ab1238500883e4a2d2bd98f812386c2"
Sep 11 12:10:12.175735 ip-10-0-135-16 kubenswrapper[2575]: E0911 12:10:12.175702    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e73d47aacef629f907185253673fb8174ab1238500883e4a2d2bd98f812386c2\": container with ID starting with e73d47aacef629f907185253673fb8174ab1238500883e4a2d2bd98f812386c2 not found: ID does not exist" containerID="e73d47aacef629f907185253673fb8174ab1238500883e4a2d2bd98f812386c2"
Sep 11 12:10:12.175813 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:12.175752    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e73d47aacef629f907185253673fb8174ab1238500883e4a2d2bd98f812386c2"} err="failed to get container status \"e73d47aacef629f907185253673fb8174ab1238500883e4a2d2bd98f812386c2\": rpc error: code = NotFound desc = could not find container \"e73d47aacef629f907185253673fb8174ab1238500883e4a2d2bd98f812386c2\": container with ID starting with e73d47aacef629f907185253673fb8174ab1238500883e4a2d2bd98f812386c2 not found: ID does not exist"
Sep 11 12:10:12.184565 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:12.184543    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"]
Sep 11 12:10:12.189440 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:12.189418    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-8f6fd9d5d-zbp79"]
Sep 11 12:10:14.060984 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:14.058882    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:14.082355 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:14.082320    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4060759e-15b8-4435-aa5b-9ffd03644ede" path="/var/lib/kubelet/pods/4060759e-15b8-4435-aa5b-9ffd03644ede/volumes"
Sep 11 12:10:24.424114 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:24.424081    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8b66479c9-6zb8s_c879f287-a93c-4ac0-9546-62a3e0405308/authorino/0.log"
Sep 11 12:10:29.764040 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:29.764006    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8bb47c4-j95vn_1493da64-9aac-4abd-a5ef-c26036b2bdc6/manager/0.log"
Sep 11 12:10:30.088977 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:30.088898    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86f769d976-2s4jb_579453ba-899a-4c77-9057-1b54a63865e3/copy-manifests/0.log"
Sep 11 12:10:30.095883 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:30.095858    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86f769d976-2s4jb_579453ba-899a-4c77-9057-1b54a63865e3/manager/0.log"
Sep 11 12:10:30.831029 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:30.831001    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-796ffbb698-gdpz8_b9d049be-d977-449c-ad4c-26954a781551/maas-api/0.log"
Sep 11 12:10:31.152897 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:31.152816    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-2fh9t_4e9ec7d6-8265-4cae-8661-057ee44fd1cc/postgres/0.log"
Sep 11 12:10:31.905453 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:31.905425    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8b66479c9-6zb8s_c879f287-a93c-4ac0-9546-62a3e0405308/authorino/0.log"
Sep 11 12:10:33.290525 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:33.290494    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-797c7d88fd-snkdh_bda5443a-7bf7-435f-a397-cbf83459fb66/kube-auth-proxy/0.log"
Sep 11 12:10:33.665479 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:33.665353    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qqpzb_66d1dfc4-81b5-49fc-bfa7-5c023917525a/payload-processing/0.log"
Sep 11 12:10:34.098134 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:34.098104    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-5v29b_3c023d59-bd7e-4d8a-884e-577d5cfb37c0/main/0.log"
Sep 11 12:10:40.450877 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:40.450848    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ntf2c/must-gather-sfwhr"]
Sep 11 12:10:40.451263 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:40.451214    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4060759e-15b8-4435-aa5b-9ffd03644ede" containerName="maas-api"
Sep 11 12:10:40.451263 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:40.451225    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4060759e-15b8-4435-aa5b-9ffd03644ede" containerName="maas-api"
Sep 11 12:10:40.451335 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:40.451301    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4060759e-15b8-4435-aa5b-9ffd03644ede" containerName="maas-api"
Sep 11 12:10:40.454426 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:40.454411    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ntf2c/must-gather-sfwhr"
Sep 11 12:10:40.459955 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:40.459931    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ntf2c\"/\"kube-root-ca.crt\""
Sep 11 12:10:40.460858 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:40.460832    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ntf2c\"/\"default-dockercfg-djvfd\""
Sep 11 12:10:40.460959 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:40.460871    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ntf2c\"/\"openshift-service-ca.crt\""
Sep 11 12:10:40.480568 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:40.480541    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ntf2c/must-gather-sfwhr"]
Sep 11 12:10:40.498330 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:40.498299    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:40.581816 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:40.581782    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e097a7a8-9323-472a-88ca-f1bcf9bc92e8-must-gather-output\") pod \"must-gather-sfwhr\" (UID: \"e097a7a8-9323-472a-88ca-f1bcf9bc92e8\") " pod="openshift-must-gather-ntf2c/must-gather-sfwhr"
Sep 11 12:10:40.581998 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:40.581840    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p45g\" (UniqueName: \"kubernetes.io/projected/e097a7a8-9323-472a-88ca-f1bcf9bc92e8-kube-api-access-9p45g\") pod \"must-gather-sfwhr\" (UID: \"e097a7a8-9323-472a-88ca-f1bcf9bc92e8\") " pod="openshift-must-gather-ntf2c/must-gather-sfwhr"
Sep 11 12:10:40.600819 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:40.600785    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log" limit=1024
Sep 11 12:10:40.682588 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:40.682553    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e097a7a8-9323-472a-88ca-f1bcf9bc92e8-must-gather-output\") pod \"must-gather-sfwhr\" (UID: \"e097a7a8-9323-472a-88ca-f1bcf9bc92e8\") " pod="openshift-must-gather-ntf2c/must-gather-sfwhr"
Sep 11 12:10:40.682817 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:40.682607    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p45g\" (UniqueName: \"kubernetes.io/projected/e097a7a8-9323-472a-88ca-f1bcf9bc92e8-kube-api-access-9p45g\") pod \"must-gather-sfwhr\" (UID: \"e097a7a8-9323-472a-88ca-f1bcf9bc92e8\") " pod="openshift-must-gather-ntf2c/must-gather-sfwhr"
Sep 11 12:10:40.682976 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:40.682955    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e097a7a8-9323-472a-88ca-f1bcf9bc92e8-must-gather-output\") pod \"must-gather-sfwhr\" (UID: \"e097a7a8-9323-472a-88ca-f1bcf9bc92e8\") " pod="openshift-must-gather-ntf2c/must-gather-sfwhr"
Sep 11 12:10:40.691777 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:40.691740    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p45g\" (UniqueName: \"kubernetes.io/projected/e097a7a8-9323-472a-88ca-f1bcf9bc92e8-kube-api-access-9p45g\") pod \"must-gather-sfwhr\" (UID: \"e097a7a8-9323-472a-88ca-f1bcf9bc92e8\") " pod="openshift-must-gather-ntf2c/must-gather-sfwhr"
Sep 11 12:10:40.764038 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:40.764012    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ntf2c/must-gather-sfwhr"
Sep 11 12:10:41.038151 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:41.038126    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ntf2c/must-gather-sfwhr"]
Sep 11 12:10:41.040353 ip-10-0-135-16 kubenswrapper[2575]: W0911 12:10:41.040329    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode097a7a8_9323_472a_88ca_f1bcf9bc92e8.slice/crio-2e792b65130a11760e5c8730389940936a1a5572b66a842b27f03cdac29c532e WatchSource:0}: Error finding container 2e792b65130a11760e5c8730389940936a1a5572b66a842b27f03cdac29c532e: Status 404 returned error can't find the container with id 2e792b65130a11760e5c8730389940936a1a5572b66a842b27f03cdac29c532e
Sep 11 12:10:41.309739 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:41.309627    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ntf2c/must-gather-sfwhr" event={"ID":"e097a7a8-9323-472a-88ca-f1bcf9bc92e8","Type":"ContainerStarted","Data":"2e792b65130a11760e5c8730389940936a1a5572b66a842b27f03cdac29c532e"}
Sep 11 12:10:42.320344 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:42.320302    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ntf2c/must-gather-sfwhr" event={"ID":"e097a7a8-9323-472a-88ca-f1bcf9bc92e8","Type":"ContainerStarted","Data":"9a939ddd7b120e8ccd838745dfde0bf6fd550cdd96918bdd25c0c53f2bd8870d"}
Sep 11 12:10:42.320882 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:42.320377    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ntf2c/must-gather-sfwhr" event={"ID":"e097a7a8-9323-472a-88ca-f1bcf9bc92e8","Type":"ContainerStarted","Data":"2772764808ce1fe8b0e8f087800fe73a1b2dccfc53362b36aac82b05b8b2f1af"}
Sep 11 12:10:42.346242 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:42.346185    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ntf2c/must-gather-sfwhr" podStartSLOduration=1.416865793 podStartE2EDuration="2.346164564s" totalImagesPullingTime="929.298771ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 12:10:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:10:42.344167229 +0000 UTC m=+2182.764773259" watchObservedRunningTime="2026-09-11 12:10:42.346164564 +0000 UTC m=+2182.766770592"
Sep 11 12:10:43.839147 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:43.839116    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nhrgs_e58ed647-be4f-4e08-b5e9-d919687aec38/global-pull-secret-syncer/0.log"
Sep 11 12:10:43.900978 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:43.900943    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6tqm4_412f846c-0794-451d-b9ed-300409e19f26/konnectivity-agent/0.log"
Sep 11 12:10:44.003931 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:44.003905    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-16.ec2.internal_a7293eba94400e360077848fce69243b/haproxy/0.log"
Sep 11 12:10:48.522426 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:48.522386    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8b66479c9-6zb8s_c879f287-a93c-4ac0-9546-62a3e0405308/authorino/0.log"
Sep 11 12:10:50.540525 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:50.540480    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf/alertmanager/0.log"
Sep 11 12:10:50.568199 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:50.568163    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf/config-reloader/0.log"
Sep 11 12:10:50.609768 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:50.609676    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf/kube-rbac-proxy-web/0.log"
Sep 11 12:10:50.639981 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:50.639953    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf/kube-rbac-proxy/0.log"
Sep 11 12:10:50.666348 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:50.666322    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf/kube-rbac-proxy-metric/0.log"
Sep 11 12:10:50.706685 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:50.706588    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf/prom-label-proxy/0.log"
Sep 11 12:10:50.758887 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:50.758857    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1b1d253b-c33f-4fa5-a26a-0f4d66bb72cf/init-config-reloader/0.log"
Sep 11 12:10:50.846121 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:50.846085    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-hz459_840bc49f-a645-4e7b-a924-0a26013e9882/kube-state-metrics/0.log"
Sep 11 12:10:50.877700 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:50.876425    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-hz459_840bc49f-a645-4e7b-a924-0a26013e9882/kube-rbac-proxy-main/0.log"
Sep 11 12:10:50.906375 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:50.906347    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-hz459_840bc49f-a645-4e7b-a924-0a26013e9882/kube-rbac-proxy-self/0.log"
Sep 11 12:10:50.957384 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:50.957324    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7c76d5974f-5ntt7_d62d9350-8ff9-4bb5-b38d-4145e6ecf12e/metrics-server/0.log"
Sep 11 12:10:51.019880 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:51.019853    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-dmr5s_f77966fc-1af3-4d4c-92b7-3e1d50372792/monitoring-plugin/0.log"
Sep 11 12:10:51.282379 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:51.282352    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r8742_271013ff-ead3-4ebc-aa2e-cfc82bede0d7/node-exporter/0.log"
Sep 11 12:10:51.312523 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:51.312495    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r8742_271013ff-ead3-4ebc-aa2e-cfc82bede0d7/kube-rbac-proxy/0.log"
Sep 11 12:10:51.342440 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:51.342415    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r8742_271013ff-ead3-4ebc-aa2e-cfc82bede0d7/init-textfile/0.log"
Sep 11 12:10:51.376525 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:51.376501    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-wvflm_750518f9-62fd-4788-a9d4-2794dbcdc844/kube-rbac-proxy-main/0.log"
Sep 11 12:10:51.419124 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:51.419098    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-wvflm_750518f9-62fd-4788-a9d4-2794dbcdc844/kube-rbac-proxy-self/0.log"
Sep 11 12:10:51.463378 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:51.463348    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-wvflm_750518f9-62fd-4788-a9d4-2794dbcdc844/openshift-state-metrics/0.log"
Sep 11 12:10:51.530326 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:51.529662    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6172b2cd-5c4a-4ecf-9aff-2cb30284ef01/prometheus/0.log"
Sep 11 12:10:51.587608 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:51.587515    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6172b2cd-5c4a-4ecf-9aff-2cb30284ef01/config-reloader/0.log"
Sep 11 12:10:51.641891 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:51.641864    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6172b2cd-5c4a-4ecf-9aff-2cb30284ef01/thanos-sidecar/0.log"
Sep 11 12:10:51.677083 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:51.677060    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6172b2cd-5c4a-4ecf-9aff-2cb30284ef01/kube-rbac-proxy-web/0.log"
Sep 11 12:10:51.707396 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:51.707363    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6172b2cd-5c4a-4ecf-9aff-2cb30284ef01/kube-rbac-proxy/0.log"
Sep 11 12:10:51.750195 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:51.750097    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6172b2cd-5c4a-4ecf-9aff-2cb30284ef01/kube-rbac-proxy-thanos/0.log"
Sep 11 12:10:51.787115 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:51.787064    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6172b2cd-5c4a-4ecf-9aff-2cb30284ef01/init-config-reloader/0.log"
Sep 11 12:10:51.825462 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:51.825435    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-tkcrj_8c467a0e-7b75-45ed-9195-e659a94a676b/prometheus-operator/0.log"
Sep 11 12:10:51.853246 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:51.853171    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-tkcrj_8c467a0e-7b75-45ed-9195-e659a94a676b/kube-rbac-proxy/0.log"
Sep 11 12:10:51.966061 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:51.966030    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5c6968755b-x2kn6_addb1856-5092-45ff-81c2-9a0b0082eb42/telemeter-client/0.log"
Sep 11 12:10:52.071681 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:52.071651    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5c6968755b-x2kn6_addb1856-5092-45ff-81c2-9a0b0082eb42/reload/0.log"
Sep 11 12:10:52.148131 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:52.148054    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5c6968755b-x2kn6_addb1856-5092-45ff-81c2-9a0b0082eb42/kube-rbac-proxy/0.log"
Sep 11 12:10:52.205698 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:52.205672    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5978f55466-qxmnm_45118d7e-8d89-47ed-aa1a-47d58f96bb73/thanos-query/0.log"
Sep 11 12:10:52.242951 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:52.242905    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5978f55466-qxmnm_45118d7e-8d89-47ed-aa1a-47d58f96bb73/kube-rbac-proxy-web/0.log"
Sep 11 12:10:52.274947 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:52.274918    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5978f55466-qxmnm_45118d7e-8d89-47ed-aa1a-47d58f96bb73/kube-rbac-proxy/0.log"
Sep 11 12:10:52.310142 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:52.310107    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5978f55466-qxmnm_45118d7e-8d89-47ed-aa1a-47d58f96bb73/prom-label-proxy/0.log"
Sep 11 12:10:52.341941 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:52.341836    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5978f55466-qxmnm_45118d7e-8d89-47ed-aa1a-47d58f96bb73/kube-rbac-proxy-rules/0.log"
Sep 11 12:10:52.392028 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:52.392002    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5978f55466-qxmnm_45118d7e-8d89-47ed-aa1a-47d58f96bb73/kube-rbac-proxy-metrics/0.log"
Sep 11 12:10:54.497359 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:54.497323    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25"]
Sep 11 12:10:54.502778 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:54.502750    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25"
Sep 11 12:10:54.528507 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:54.528471    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25"]
Sep 11 12:10:54.647797 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:54.647757    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca4966b1-4127-4b28-8a37-71a7457c4f8a-lib-modules\") pod \"perf-node-gather-daemonset-2mz25\" (UID: \"ca4966b1-4127-4b28-8a37-71a7457c4f8a\") " pod="openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25"
Sep 11 12:10:54.647986 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:54.647830    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ca4966b1-4127-4b28-8a37-71a7457c4f8a-proc\") pod \"perf-node-gather-daemonset-2mz25\" (UID: \"ca4966b1-4127-4b28-8a37-71a7457c4f8a\") " pod="openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25"
Sep 11 12:10:54.648059 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:54.647990    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca4966b1-4127-4b28-8a37-71a7457c4f8a-sys\") pod \"perf-node-gather-daemonset-2mz25\" (UID: \"ca4966b1-4127-4b28-8a37-71a7457c4f8a\") " pod="openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25"
Sep 11 12:10:54.648059 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:54.648015    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzt76\" (UniqueName: \"kubernetes.io/projected/ca4966b1-4127-4b28-8a37-71a7457c4f8a-kube-api-access-hzt76\") pod \"perf-node-gather-daemonset-2mz25\" (UID: \"ca4966b1-4127-4b28-8a37-71a7457c4f8a\") " pod="openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25"
Sep 11 12:10:54.648157 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:54.648054    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ca4966b1-4127-4b28-8a37-71a7457c4f8a-podres\") pod \"perf-node-gather-daemonset-2mz25\" (UID: \"ca4966b1-4127-4b28-8a37-71a7457c4f8a\") " pod="openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25"
Sep 11 12:10:54.749198 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:54.749106    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca4966b1-4127-4b28-8a37-71a7457c4f8a-sys\") pod \"perf-node-gather-daemonset-2mz25\" (UID: \"ca4966b1-4127-4b28-8a37-71a7457c4f8a\") " pod="openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25"
Sep 11 12:10:54.749198 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:54.749152    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzt76\" (UniqueName: \"kubernetes.io/projected/ca4966b1-4127-4b28-8a37-71a7457c4f8a-kube-api-access-hzt76\") pod \"perf-node-gather-daemonset-2mz25\" (UID: \"ca4966b1-4127-4b28-8a37-71a7457c4f8a\") " pod="openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25"
Sep 11 12:10:54.749439 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:54.749257    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ca4966b1-4127-4b28-8a37-71a7457c4f8a-podres\") pod \"perf-node-gather-daemonset-2mz25\" (UID: \"ca4966b1-4127-4b28-8a37-71a7457c4f8a\") " pod="openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25"
Sep 11 12:10:54.749439 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:54.749343    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca4966b1-4127-4b28-8a37-71a7457c4f8a-sys\") pod \"perf-node-gather-daemonset-2mz25\" (UID: \"ca4966b1-4127-4b28-8a37-71a7457c4f8a\") " pod="openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25"
Sep 11 12:10:54.749439 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:54.749344    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca4966b1-4127-4b28-8a37-71a7457c4f8a-lib-modules\") pod \"perf-node-gather-daemonset-2mz25\" (UID: \"ca4966b1-4127-4b28-8a37-71a7457c4f8a\") " pod="openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25"
Sep 11 12:10:54.749439 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:54.749397    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ca4966b1-4127-4b28-8a37-71a7457c4f8a-podres\") pod \"perf-node-gather-daemonset-2mz25\" (UID: \"ca4966b1-4127-4b28-8a37-71a7457c4f8a\") " pod="openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25"
Sep 11 12:10:54.749439 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:54.749412    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ca4966b1-4127-4b28-8a37-71a7457c4f8a-proc\") pod \"perf-node-gather-daemonset-2mz25\" (UID: \"ca4966b1-4127-4b28-8a37-71a7457c4f8a\") " pod="openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25"
Sep 11 12:10:54.749668 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:54.749460    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ca4966b1-4127-4b28-8a37-71a7457c4f8a-proc\") pod \"perf-node-gather-daemonset-2mz25\" (UID: \"ca4966b1-4127-4b28-8a37-71a7457c4f8a\") " pod="openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25"
Sep 11 12:10:54.749668 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:54.749464    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca4966b1-4127-4b28-8a37-71a7457c4f8a-lib-modules\") pod \"perf-node-gather-daemonset-2mz25\" (UID: \"ca4966b1-4127-4b28-8a37-71a7457c4f8a\") " pod="openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25"
Sep 11 12:10:54.758890 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:54.758838    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzt76\" (UniqueName: \"kubernetes.io/projected/ca4966b1-4127-4b28-8a37-71a7457c4f8a-kube-api-access-hzt76\") pod \"perf-node-gather-daemonset-2mz25\" (UID: \"ca4966b1-4127-4b28-8a37-71a7457c4f8a\") " pod="openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25"
Sep 11 12:10:54.818934 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:54.818897    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25"
Sep 11 12:10:55.067465 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:55.067396    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6c66799df7-fb6z8_7a10215c-ab9e-4c6c-a728-33503b6a2281/console/0.log"
Sep 11 12:10:55.117861 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:55.117832    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25"]
Sep 11 12:10:55.413962 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:55.413865    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25" event={"ID":"ca4966b1-4127-4b28-8a37-71a7457c4f8a","Type":"ContainerStarted","Data":"c1b757ca3560a5ae2b47117ca61702d5c9ab710ae3599e780315206673102abf"}
Sep 11 12:10:55.414116 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:55.413971    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25" event={"ID":"ca4966b1-4127-4b28-8a37-71a7457c4f8a","Type":"ContainerStarted","Data":"66a5bee9b5e86017541a12f10481a0271850030bce49e856ca2e72a2a350d31c"}
Sep 11 12:10:55.414116 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:55.414002    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25"
Sep 11 12:10:55.437874 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:55.437814    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25" podStartSLOduration=1.437796287 podStartE2EDuration="1.437796287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 12:10:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 12:10:55.435260485 +0000 UTC m=+2195.855866534" watchObservedRunningTime="2026-09-11 12:10:55.437796287 +0000 UTC m=+2195.858402313"
Sep 11 12:10:56.999558 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:56.999504    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-k6szw_2561a637-628e-41ca-a07c-18ba49d56cb2/dns/0.log"
Sep 11 12:10:57.029830 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:57.029788    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-k6szw_2561a637-628e-41ca-a07c-18ba49d56cb2/kube-rbac-proxy/0.log"
Sep 11 12:10:57.129309 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:57.129271    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vf4g9_88a0717f-18e4-40d9-914c-96ccaaf0e05f/dns-node-resolver/0.log"
Sep 11 12:10:57.836555 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:57.836529    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lx4vv_9470c69c-fd20-49fc-8b0f-e02d3010e1bc/node-ca/0.log"
Sep 11 12:10:59.161763 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:59.161732    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-797c7d88fd-snkdh_bda5443a-7bf7-435f-a397-cbf83459fb66/kube-auth-proxy/0.log"
Sep 11 12:10:59.285817 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:10:59.285783    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qqpzb_66d1dfc4-81b5-49fc-bfa7-5c023917525a/payload-processing/0.log"
Sep 11 12:11:00.349593 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:11:00.349568    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wvb2j_29f9b723-dc90-4bf0-bfcb-42bff332e469/serve-healthcheck-canary/0.log"
Sep 11 12:11:01.099329 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:11:01.099296    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ngm7d_d070541c-7f68-4350-a203-048ec6f3cf69/kube-rbac-proxy/0.log"
Sep 11 12:11:01.132114 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:11:01.132078    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ngm7d_d070541c-7f68-4350-a203-048ec6f3cf69/exporter/0.log"
Sep 11 12:11:01.160171 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:11:01.160140    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ngm7d_d070541c-7f68-4350-a203-048ec6f3cf69/extractor/0.log"
Sep 11 12:11:01.434407 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:11:01.434331    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ntf2c/perf-node-gather-daemonset-2mz25"
Sep 11 12:11:03.824065 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:11:03.824034    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8bb47c4-j95vn_1493da64-9aac-4abd-a5ef-c26036b2bdc6/manager/0.log"
Sep 11 12:11:03.972626 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:11:03.972597    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86f769d976-2s4jb_579453ba-899a-4c77-9057-1b54a63865e3/manager/0.log"
Sep 11 12:11:04.014288 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:11:04.014264    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86f769d976-2s4jb_579453ba-899a-4c77-9057-1b54a63865e3/copy-manifests/0.log"
Sep 11 12:11:05.454050 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:11:05.454019    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-797c7d88fd-snkdh_bda5443a-7bf7-435f-a397-cbf83459fb66/kube-auth-proxy/0.log"
Sep 11 12:11:05.574614 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:11:05.574584    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qqpzb_66d1dfc4-81b5-49fc-bfa7-5c023917525a/payload-processing/0.log"
Sep 11 12:11:05.653145 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:11:05.653099    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-77956949cf-bjlrz_9f1eb6de-66ef-49f8-a33f-363cc1a20412/manager/0.log"
Sep 11 12:11:05.695354 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:11:05.695292    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-mjz84_889c079f-f425-40c9-9669-e141d2b60c46/openshift-lws-operator/0.log"
Sep 11 12:11:07.396274 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:11:07.396244    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-797c7d88fd-snkdh_bda5443a-7bf7-435f-a397-cbf83459fb66/kube-auth-proxy/0.log"
Sep 11 12:11:07.491743 ip-10-0-135-16 kubenswrapper[2575]: I0911 12:11:07.491710    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qqpzb_66d1dfc4-81b5-49fc-bfa7-5c023917525a/payload-processing/0.log"