Sep 15 20:57:43.336807 ip-10-0-138-221 systemd[1]: Starting Kubernetes Kubelet...
Sep 15 20:57:43.766649 ip-10-0-138-221 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 20:57:43.766649 ip-10-0-138-221 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 15 20:57:43.766649 ip-10-0-138-221 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 20:57:43.766649 ip-10-0-138-221 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 15 20:57:43.766649 ip-10-0-138-221 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 20:57:43.767355 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.766715    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 15 20:57:43.770635 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770618    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:57:43.770635 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770634    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:57:43.770708 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770640    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:57:43.770708 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770643    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:57:43.770708 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770646    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:57:43.770708 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770650    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:57:43.770708 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770653    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:57:43.770708 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770656    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:57:43.770708 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770659    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:57:43.770708 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770662    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:57:43.770708 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770664    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:57:43.770708 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770667    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:57:43.770708 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770670    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:57:43.770708 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770679    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:57:43.770708 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770682    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:57:43.770708 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770685    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:57:43.770708 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770690    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:57:43.770708 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770693    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:57:43.770708 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770696    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:57:43.770708 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770699    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:57:43.770708 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770702    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:57:43.771191 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770705    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:57:43.771191 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770708    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:57:43.771191 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770711    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:57:43.771191 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770714    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:57:43.771191 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770717    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:57:43.771191 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770721    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:57:43.771191 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770724    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:57:43.771191 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770726    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:57:43.771191 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770729    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:57:43.771191 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770733    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:57:43.771191 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770736    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:57:43.771191 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770739    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:57:43.771191 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770741    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:57:43.771191 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770744    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:57:43.771191 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770746    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:57:43.771191 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770749    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:57:43.771191 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770753    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:57:43.771191 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770756    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:57:43.771191 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770760    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:57:43.771191 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770762    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:57:43.771969 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770765    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:57:43.771969 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770767    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:57:43.771969 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770770    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:57:43.771969 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770772    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:57:43.771969 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770774    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:57:43.771969 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770780    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:57:43.771969 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770782    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:57:43.771969 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770784    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:57:43.771969 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770787    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:57:43.771969 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770790    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:57:43.771969 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770793    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:57:43.771969 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770796    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:57:43.771969 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770798    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:57:43.771969 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770802    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:57:43.771969 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770805    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:57:43.771969 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770808    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:57:43.771969 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770810    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:57:43.771969 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770813    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:57:43.771969 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770816    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:57:43.771969 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770818    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:57:43.772483 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770820    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:57:43.772483 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770823    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:57:43.772483 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770826    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:57:43.772483 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770828    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:57:43.772483 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770831    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:57:43.772483 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770834    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:57:43.772483 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770836    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:57:43.772483 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770838    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:57:43.772483 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770841    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:57:43.772483 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770843    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:57:43.772483 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770847    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:57:43.772483 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770849    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:57:43.772483 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770852    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:57:43.772483 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770855    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:57:43.772483 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770857    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:57:43.772483 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770860    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:57:43.772483 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770862    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:57:43.772483 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770865    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:57:43.772483 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770870    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:57:43.772483 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770873    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:57:43.772992 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770875    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:57:43.772992 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770878    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:57:43.772992 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770880    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:57:43.772992 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770882    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:57:43.772992 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770885    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:57:43.772992 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770889    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:57:43.772992 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770891    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:57:43.772992 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770894    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:57:43.772992 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770896    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:57:43.772992 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770899    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:57:43.772992 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770901    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:57:43.772992 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770903    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:57:43.772992 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770906    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:57:43.772992 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770909    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:57:43.772992 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770911    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:57:43.772992 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770913    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:57:43.772992 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770916    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:57:43.772992 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.770918    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:57:43.774996 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.774980    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:57:43.774996 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.774997    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:57:43.775066 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775001    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:57:43.775066 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775006    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:57:43.775066 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775011    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:57:43.775066 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775014    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:57:43.775066 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775017    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:57:43.775066 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775020    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:57:43.775066 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775023    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:57:43.775066 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775026    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:57:43.775066 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775028    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:57:43.775066 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775031    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:57:43.775066 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775033    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:57:43.775066 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775036    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:57:43.775066 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775039    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:57:43.775066 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775046    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:57:43.775066 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775049    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:57:43.775066 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775051    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:57:43.775066 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775054    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:57:43.775066 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775057    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:57:43.775066 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775059    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:57:43.775066 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775064    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:57:43.775586 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775067    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:57:43.775586 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775070    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:57:43.775586 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775073    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:57:43.775586 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775075    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:57:43.775586 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775078    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:57:43.775586 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775081    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:57:43.775586 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775083    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:57:43.775586 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775086    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:57:43.775586 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775089    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:57:43.775586 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775091    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:57:43.775586 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775094    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:57:43.775586 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775096    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:57:43.775586 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775099    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:57:43.775586 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775101    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:57:43.775586 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775104    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:57:43.775586 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775106    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:57:43.775586 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775109    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:57:43.775586 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775111    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:57:43.775586 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775114    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:57:43.775586 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775117    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:57:43.776122 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775119    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:57:43.776122 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775122    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:57:43.776122 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775125    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:57:43.776122 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775127    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:57:43.776122 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775130    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:57:43.776122 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775133    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:57:43.776122 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775135    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:57:43.776122 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775137    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:57:43.776122 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775140    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:57:43.776122 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775143    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:57:43.776122 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775145    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:57:43.776122 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775148    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:57:43.776122 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775150    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:57:43.776122 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775154    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:57:43.776122 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775156    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:57:43.776122 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775159    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:57:43.776122 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775162    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:57:43.776122 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775164    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:57:43.776122 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775167    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:57:43.776616 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775169    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:57:43.776616 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775172    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:57:43.776616 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775174    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:57:43.776616 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775177    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:57:43.776616 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775179    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:57:43.776616 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775182    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:57:43.776616 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775184    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:57:43.776616 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775187    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:57:43.776616 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775190    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:57:43.776616 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775193    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:57:43.776616 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775195    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:57:43.776616 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775198    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:57:43.776616 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775200    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:57:43.776616 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775203    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:57:43.776616 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775205    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:57:43.776616 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775208    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:57:43.776616 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775211    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:57:43.776616 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775214    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:57:43.776616 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775217    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:57:43.777079 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775220    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:57:43.777079 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775223    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:57:43.777079 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775226    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:57:43.777079 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775231    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:57:43.777079 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775234    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:57:43.777079 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775237    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:57:43.777079 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775240    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:57:43.777079 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775244    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:57:43.777079 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775248    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:57:43.777079 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775251    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:57:43.777079 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775253    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:57:43.777079 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775256    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:57:43.777079 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775259    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:57:43.777079 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775262    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:57:43.777079 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775265    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:57:43.777079 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775267    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:57:43.777079 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775270    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:57:43.777079 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775272    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:57:43.777079 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775275    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:57:43.777079 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775696    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:57:43.777591 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775702    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:57:43.777591 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775705    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:57:43.777591 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775707    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:57:43.777591 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775710    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:57:43.777591 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775713    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:57:43.777591 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775715    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:57:43.777591 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775718    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:57:43.777591 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775721    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:57:43.777591 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775723    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:57:43.777591 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775726    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:57:43.777591 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775729    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:57:43.777591 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775732    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:57:43.777591 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775734    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:57:43.777591 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775737    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:57:43.777591 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775740    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:57:43.777591 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775743    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:57:43.777591 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775746    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:57:43.777591 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775748    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:57:43.777591 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775751    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:57:43.777591 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775756    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:57:43.778153 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775760    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:57:43.778153 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775763    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:57:43.778153 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775766    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:57:43.778153 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775768    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:57:43.778153 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775771    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:57:43.778153 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775774    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:57:43.778153 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775776    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:57:43.778153 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775779    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:57:43.778153 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775782    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:57:43.778153 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775784    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:57:43.778153 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775787    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:57:43.778153 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775790    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:57:43.778153 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775792    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:57:43.778153 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775795    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:57:43.778153 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775797    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:57:43.778153 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775800    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:57:43.778153 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775802    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:57:43.778153 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775805    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:57:43.778153 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775807    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:57:43.778153 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775810    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:57:43.778677 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775812    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:57:43.778677 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775815    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:57:43.778677 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775817    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:57:43.778677 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775820    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:57:43.778677 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775823    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:57:43.778677 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775826    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:57:43.778677 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775829    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:57:43.778677 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775831    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:57:43.778677 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775834    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:57:43.778677 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775837    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:57:43.778677 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775840    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:57:43.778677 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775843    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:57:43.778677 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775846    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:57:43.778677 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775849    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:57:43.778677 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775851    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:57:43.778677 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775854    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:57:43.778677 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775857    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:57:43.778677 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775860    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:57:43.778677 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775862    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:57:43.778677 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775865    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:57:43.779167 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775868    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:57:43.779167 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775870    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:57:43.779167 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775873    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:57:43.779167 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775875    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:57:43.779167 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775878    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:57:43.779167 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775880    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:57:43.779167 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775883    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:57:43.779167 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775885    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:57:43.779167 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775888    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:57:43.779167 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775890    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:57:43.779167 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775893    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:57:43.779167 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775895    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:57:43.779167 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775898    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:57:43.779167 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775900    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:57:43.779167 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775903    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:57:43.779167 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775905    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:57:43.779167 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775909    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:57:43.779167 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775911    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:57:43.779167 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775914    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:57:43.779650 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775916    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:57:43.779650 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775921    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:57:43.779650 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775925    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:57:43.779650 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775927    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:57:43.779650 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775931    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:57:43.779650 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775934    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:57:43.779650 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775937    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:57:43.779650 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775939    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:57:43.779650 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775942    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:57:43.779650 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775946    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:57:43.779650 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775949    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:57:43.779650 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775952    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:57:43.779650 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775954    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:57:43.779650 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775957    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:57:43.779650 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775959    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:57:43.779650 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775962    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:57:43.779650 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775964    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:57:43.779650 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775967    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:57:43.779650 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.775969    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776047    2564 flags.go:64] FLAG: --address="0.0.0.0"
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776101    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776109    2564 flags.go:64] FLAG: --anonymous-auth="true"
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776118    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776122    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776125    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776130    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776134    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776137    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776140    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776144    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776147    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776150    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776153    2564 flags.go:64] FLAG: --cgroup-root=""
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776156    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776159    2564 flags.go:64] FLAG: --client-ca-file=""
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776163    2564 flags.go:64] FLAG: --cloud-provider="external"
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776167    2564 flags.go:64] FLAG: --cluster-dns="[]"
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776171    2564 flags.go:64] FLAG: --cluster-domain=""
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776174    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776177    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776180    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776184    2564 flags.go:64] FLAG: --container-log-max-files="5"
Sep 15 20:57:43.780118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776187    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776190    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776193    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776197    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776200    2564 flags.go:64] FLAG: --contention-profiling="false"
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776202    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776205    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776209    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776211    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776215    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776218    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776221    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776224    2564 flags.go:64] FLAG: --enable-load-reader="false"
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776227    2564 flags.go:64] FLAG: --enable-server="true"
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776230    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776234    2564 flags.go:64] FLAG: --event-burst="100"
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776237    2564 flags.go:64] FLAG: --event-qps="50"
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776240    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776243    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776246    2564 flags.go:64] FLAG: --eviction-hard=""
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776250    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776253    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776256    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776259    2564 flags.go:64] FLAG: --eviction-soft=""
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776261    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 15 20:57:43.780706 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776265    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776269    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776272    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776275    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776278    2564 flags.go:64] FLAG: --fail-swap-on="true"
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776281    2564 flags.go:64] FLAG: --feature-gates=""
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776285    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776288    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776291    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776294    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776297    2564 flags.go:64] FLAG: --healthz-port="10248"
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776300    2564 flags.go:64] FLAG: --help="false"
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776303    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-138-221.ec2.internal"
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776306    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776309    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776312    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776315    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776319    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776322    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776324    2564 flags.go:64] FLAG: --image-service-endpoint=""
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776327    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776330    2564 flags.go:64] FLAG: --kube-api-burst="100"
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776333    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776336    2564 flags.go:64] FLAG: --kube-api-qps="50"
Sep 15 20:57:43.781367 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776339    2564 flags.go:64] FLAG: --kube-reserved=""
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776341    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776344    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776347    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776349    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776352    2564 flags.go:64] FLAG: --lock-file=""
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776355    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776357    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776360    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776366    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776370    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776373    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776375    2564 flags.go:64] FLAG: --logging-format="text"
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776378    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776381    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776384    2564 flags.go:64] FLAG: --manifest-url=""
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776386    2564 flags.go:64] FLAG: --manifest-url-header=""
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776391    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776394    2564 flags.go:64] FLAG: --max-open-files="1000000"
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776398    2564 flags.go:64] FLAG: --max-pods="110"
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776401    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776417    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776421    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776424    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776427    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 15 20:57:43.782046 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776430    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776433    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776441    2564 flags.go:64] FLAG: --node-status-max-images="50"
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776444    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776447    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776450    2564 flags.go:64] FLAG: --pod-cidr=""
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776453    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776459    2564 flags.go:64] FLAG: --pod-manifest-path=""
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776462    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776465    2564 flags.go:64] FLAG: --pods-per-core="0"
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776468    2564 flags.go:64] FLAG: --port="10250"
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776471    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776474    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03fc6a8dc32b1b44b"
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776477    2564 flags.go:64] FLAG: --qos-reserved=""
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776480    2564 flags.go:64] FLAG: --read-only-port="10255"
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776483    2564 flags.go:64] FLAG: --register-node="true"
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776503    2564 flags.go:64] FLAG: --register-with-taints=""
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776510    2564 flags.go:64] FLAG: --registry-burst="10"
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776515    2564 flags.go:64] FLAG: --registry-qps="5"
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776519    2564 flags.go:64] FLAG: --reserved-cpus=""
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776522    2564 flags.go:64] FLAG: --reserved-memory=""
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776526    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776528    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776531    2564 flags.go:64] FLAG: --rotate-certificates="false"
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776534    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 15 20:57:43.782685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776537    2564 flags.go:64] FLAG: --runonce="false"
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776539    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776543    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776546    2564 flags.go:64] FLAG: --seccomp-default="false"
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776549    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776552    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776554    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776557    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776560    2564 flags.go:64] FLAG: --storage-driver-password="root"
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776563    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776566    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776568    2564 flags.go:64] FLAG: --storage-driver-user="root"
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776571    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776574    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776576    2564 flags.go:64] FLAG: --system-cgroups=""
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776579    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776584    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776587    2564 flags.go:64] FLAG: --tls-cert-file=""
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776589    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776593    2564 flags.go:64] FLAG: --tls-min-version=""
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776596    2564 flags.go:64] FLAG: --tls-private-key-file=""
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776598    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776601    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776606    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776609    2564 flags.go:64] FLAG: --v="2"
Sep 15 20:57:43.783351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776613    2564 flags.go:64] FLAG: --version="false"
Sep 15 20:57:43.784425 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776619    2564 flags.go:64] FLAG: --vmodule=""
Sep 15 20:57:43.784425 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776622    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 15 20:57:43.784425 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.776626    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 15 20:57:43.784425 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776750    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:57:43.784425 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776754    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:57:43.784425 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776757    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:57:43.784425 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776761    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:57:43.784425 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776763    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:57:43.784425 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776766    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:57:43.784425 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776769    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:57:43.784425 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776771    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:57:43.784425 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776774    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:57:43.784425 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776776    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:57:43.784425 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776780    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:57:43.784425 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776782    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:57:43.784425 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776785    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:57:43.784425 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776787    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:57:43.784425 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776791    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:57:43.784425 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776795    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:57:43.784425 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776799    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:57:43.785284 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776802    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:57:43.785284 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776805    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:57:43.785284 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776808    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:57:43.785284 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776811    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:57:43.785284 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776814    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:57:43.785284 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776817    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:57:43.785284 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776820    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:57:43.785284 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776823    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:57:43.785284 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776826    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:57:43.785284 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776830    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:57:43.785284 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776833    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:57:43.785284 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776835    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:57:43.785284 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776838    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:57:43.785284 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776844    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:57:43.785284 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776847    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:57:43.785284 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776850    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:57:43.785284 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776852    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:57:43.785284 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776855    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:57:43.785284 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776857    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:57:43.785284 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776860    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:57:43.785962 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776862    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:57:43.785962 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776865    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:57:43.785962 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776867    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:57:43.785962 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776869    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:57:43.785962 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776872    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:57:43.785962 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776875    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:57:43.785962 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776877    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:57:43.785962 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776880    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:57:43.785962 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776884    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:57:43.785962 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776887    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:57:43.785962 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776890    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:57:43.785962 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776892    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:57:43.785962 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776895    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:57:43.785962 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776897    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:57:43.785962 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776900    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:57:43.785962 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776902    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:57:43.785962 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776905    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:57:43.785962 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776908    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:57:43.785962 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776910    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:57:43.786484 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776913    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:57:43.786484 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776915    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:57:43.786484 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776919    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:57:43.786484 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776922    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:57:43.786484 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776924    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:57:43.786484 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776926    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:57:43.786484 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776930    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:57:43.786484 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776933    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:57:43.786484 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776935    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:57:43.786484 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776938    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:57:43.786484 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776940    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:57:43.786484 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776943    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:57:43.786484 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776945    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:57:43.786484 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776948    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:57:43.786484 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776951    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:57:43.786484 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776953    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:57:43.786484 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776956    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:57:43.786484 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776959    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:57:43.786484 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776963    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:57:43.787338 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776966    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:57:43.787338 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776969    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:57:43.787338 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776973    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:57:43.787338 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776976    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:57:43.787338 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776978    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:57:43.787338 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776981    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:57:43.787338 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776984    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:57:43.787338 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776986    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:57:43.787338 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776989    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:57:43.787338 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776991    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:57:43.787338 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776994    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:57:43.787338 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776996    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:57:43.787338 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.776999    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:57:43.787338 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.777001    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:57:43.787338 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.777004    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:57:43.787338 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.777007    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:57:43.787338 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.777010    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:57:43.787338 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.777013    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:57:43.787338 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.777015    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:57:43.787338 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.777018    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:57:43.788133 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.777020    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:57:43.788133 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.777024    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:57:43.788133 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.777026    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:57:43.788133 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.777029    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:57:43.788133 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.777701    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 20:57:43.788133 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.785612    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 15 20:57:43.788133 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.785632    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 15 20:57:43.788133 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.786392    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 15 20:57:43.788133 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.786404    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 15 20:57:43.788133 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786521    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:57:43.788133 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786538    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:57:43.788133 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786544    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:57:43.788133 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786548    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:57:43.788133 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786553    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:57:43.788133 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786558    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:57:43.788605 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786563    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:57:43.788605 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786568    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:57:43.788605 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786573    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:57:43.788605 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786577    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:57:43.788605 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786581    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:57:43.788605 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786585    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:57:43.788605 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786589    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:57:43.788605 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786593    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:57:43.788605 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786598    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:57:43.788605 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786602    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:57:43.788605 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786607    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:57:43.788605 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786611    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:57:43.788605 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786615    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:57:43.788605 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786619    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:57:43.788605 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786623    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:57:43.788605 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786627    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:57:43.788605 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786631    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:57:43.788605 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786635    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:57:43.788605 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786639    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:57:43.788605 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786644    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:57:43.789428 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786649    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:57:43.789428 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786653    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:57:43.789428 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786657    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:57:43.789428 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786664    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:57:43.789428 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786668    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:57:43.789428 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786672    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:57:43.789428 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786676    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:57:43.789428 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786680    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:57:43.789428 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786684    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:57:43.789428 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786688    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:57:43.789428 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786693    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:57:43.789428 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786697    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:57:43.789428 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786701    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:57:43.789428 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786705    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:57:43.789428 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786709    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:57:43.789428 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786713    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:57:43.789428 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786717    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:57:43.789428 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786721    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:57:43.789428 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786725    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:57:43.790177 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786730    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:57:43.790177 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786734    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:57:43.790177 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786738    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:57:43.790177 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786742    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:57:43.790177 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786746    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:57:43.790177 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786750    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:57:43.790177 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786754    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:57:43.790177 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786758    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:57:43.790177 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786762    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:57:43.790177 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786767    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:57:43.790177 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786772    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:57:43.790177 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786776    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:57:43.790177 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786781    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:57:43.790177 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786785    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:57:43.790177 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786789    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:57:43.790177 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786793    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:57:43.790177 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786798    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:57:43.790177 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786805    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:57:43.790177 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786812    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:57:43.790177 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786817    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:57:43.790881 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786821    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:57:43.790881 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786825    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:57:43.790881 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786829    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:57:43.790881 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786833    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:57:43.790881 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786837    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:57:43.790881 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786842    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:57:43.790881 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786845    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:57:43.790881 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786850    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:57:43.790881 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786856    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:57:43.790881 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786862    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:57:43.790881 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786867    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:57:43.790881 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786871    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:57:43.790881 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786876    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:57:43.790881 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786880    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:57:43.790881 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786884    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:57:43.790881 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786888    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:57:43.790881 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786892    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:57:43.790881 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786896    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:57:43.790881 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786901    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:57:43.791356 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786905    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:57:43.791356 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786909    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:57:43.791356 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786914    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:57:43.791356 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786918    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:57:43.791356 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786922    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:57:43.791356 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786926    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:57:43.791356 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786930    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:57:43.791356 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786934    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:57:43.791356 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786941    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:57:43.791356 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786946    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:57:43.791356 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786951    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:57:43.791356 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786957    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:57:43.791356 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786962    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:57:43.791356 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786967    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:57:43.791356 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.786971    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:57:43.791801 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.786980    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 20:57:43.791801 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787209    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 20:57:43.791801 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787219    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 20:57:43.791801 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787224    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 20:57:43.791801 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787229    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 20:57:43.791801 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787233    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 20:57:43.791801 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787238    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 20:57:43.791801 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787242    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 20:57:43.791801 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787246    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 20:57:43.791801 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787253    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 20:57:43.791801 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787259    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 20:57:43.791801 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787264    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 20:57:43.791801 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787268    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 20:57:43.791801 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787273    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 20:57:43.792147 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787277    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 20:57:43.792147 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787282    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 20:57:43.792147 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787286    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 20:57:43.792147 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787291    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 20:57:43.792147 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787295    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 20:57:43.792147 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787299    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 20:57:43.792147 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787303    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 20:57:43.792147 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787309    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 20:57:43.792147 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787315    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 20:57:43.792147 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787320    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 20:57:43.792147 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787324    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 20:57:43.792147 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787328    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 20:57:43.792147 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787332    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 20:57:43.792147 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787336    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 20:57:43.792147 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787340    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 20:57:43.792147 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787345    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 20:57:43.792147 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787349    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 20:57:43.792147 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787353    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 20:57:43.792147 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787356    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 20:57:43.792147 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787360    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 20:57:43.792774 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787364    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 20:57:43.792774 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787368    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 20:57:43.792774 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787372    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 20:57:43.792774 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787376    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 20:57:43.792774 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787381    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 20:57:43.792774 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787386    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 20:57:43.792774 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787390    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 20:57:43.792774 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787394    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 20:57:43.792774 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787399    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 20:57:43.792774 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787403    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 20:57:43.792774 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787408    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 20:57:43.792774 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787412    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 20:57:43.792774 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787416    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 20:57:43.792774 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787420    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 20:57:43.792774 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787425    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 20:57:43.792774 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787429    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 20:57:43.792774 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787433    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 20:57:43.792774 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787437    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 20:57:43.792774 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787441    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 20:57:43.792774 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787445    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 20:57:43.793277 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787450    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 20:57:43.793277 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787454    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 20:57:43.793277 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787458    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 20:57:43.793277 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787462    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 20:57:43.793277 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787466    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 20:57:43.793277 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787471    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 20:57:43.793277 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787475    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 20:57:43.793277 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787479    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 20:57:43.793277 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787503    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 20:57:43.793277 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787508    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 20:57:43.793277 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787515    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 20:57:43.793277 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787520    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 20:57:43.793277 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787525    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 20:57:43.793277 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787529    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 20:57:43.793277 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787534    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 20:57:43.793277 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787538    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 20:57:43.793277 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787542    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 20:57:43.793277 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787547    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 20:57:43.793277 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787551    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 20:57:43.793775 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787555    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 20:57:43.793775 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787559    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 20:57:43.793775 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787563    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 20:57:43.793775 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787567    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 20:57:43.793775 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787571    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 20:57:43.793775 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787575    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 20:57:43.793775 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787578    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 20:57:43.793775 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787582    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 20:57:43.793775 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787587    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 20:57:43.793775 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787591    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 20:57:43.793775 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787595    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 20:57:43.793775 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787599    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 20:57:43.793775 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787603    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 20:57:43.793775 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787608    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 20:57:43.793775 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787612    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 20:57:43.793775 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787616    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 20:57:43.793775 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787620    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 20:57:43.793775 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787624    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 20:57:43.793775 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787628    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 20:57:43.794318 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787632    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 20:57:43.794318 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787636    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 20:57:43.794318 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787640    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 20:57:43.794318 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787645    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 20:57:43.794318 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787649    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 20:57:43.794318 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787653    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 20:57:43.794318 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787657    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 20:57:43.794318 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:43.787661    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 20:57:43.794318 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.787669    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 20:57:43.794318 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.787843    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 15 20:57:43.794318 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.790561    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 15 20:57:43.794318 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.791701    2564 server.go:1019] "Starting client certificate rotation"
Sep 15 20:57:43.794318 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.791826    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 15 20:57:43.794318 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.791875    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 15 20:57:43.794989 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.794972    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 15 20:57:43.796671 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.796633    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 15 20:57:43.808685 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.808538    2564 log.go:25] "Validated CRI v1 runtime API"
Sep 15 20:57:43.811807 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.811790    2564 log.go:25] "Validated CRI v1 image API"
Sep 15 20:57:43.812700 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.812683    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 15 20:57:43.818303 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.818282    2564 fs.go:135] Filesystem UUIDs: map[1640a980-a1b8-4c37-87dc-6010d834c97f:/dev/nvme0n1p3 7522b4fa-d6d7-44f4-9ae7-5bd97a538a61:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 15 20:57:43.818381 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.818301    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 15 20:57:43.822339 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.822320    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 15 20:57:43.824614 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.824474    2564 manager.go:216] Machine: {Timestamp:2026-09-15 20:57:43.822144983 +0000 UTC m=+0.364307620 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101399 MemoryCapacity:32812154880 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22b6ab436ec4f8fdede6f047a3e9c3 SystemUUID:ec22b6ab-436e-c4f8-fded-e6f047a3e9c3 BootID:ddb0a2f9-5e79-476a-967a-25c7ea45b6f1 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c1:00:f4:af Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c1:00:f4:af Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ce:aa:bc:57:9c:09 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812154880 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 15 20:57:43.824614 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.824609    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 15 20:57:43.824743 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.824731    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 15 20:57:43.827672 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.827645    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 15 20:57:43.827822 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.827678    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-221.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 15 20:57:43.828461 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.828451    2564 topology_manager.go:138] "Creating topology manager with none policy"
Sep 15 20:57:43.828515 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.828463    2564 container_manager_linux.go:309] "Creating device plugin manager"
Sep 15 20:57:43.828515 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.828472    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 15 20:57:43.829154 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.829143    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 15 20:57:43.830193 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.830182    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 15 20:57:43.834289 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.834276    2564 state_mem.go:36] "Initialized new in-memory state store"
Sep 15 20:57:43.834417 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.834408    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 15 20:57:43.834467 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.834439    2564 kubelet.go:486] "Attempting to sync node with API server"
Sep 15 20:57:43.834467 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.834462    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 15 20:57:43.834579 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.834478    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 15 20:57:43.834579 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.834500    2564 kubelet.go:398] "Adding apiserver pod source"
Sep 15 20:57:43.834579 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.834510    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 15 20:57:43.834917 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.834903    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 15 20:57:43.834953 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.834932    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 15 20:57:43.837262 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.837245    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 15 20:57:43.839524 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.839506    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 15 20:57:43.839578 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.839541    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 15 20:57:43.842538 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.842517    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 15 20:57:43.842580 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.842542    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 15 20:57:43.842580 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.842550    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 15 20:57:43.842580 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.842555    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 15 20:57:43.842580 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.842564    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 15 20:57:43.842580 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.842569    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 15 20:57:43.842580 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.842582    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 15 20:57:43.842736 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.842587    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 15 20:57:43.842736 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.842593    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 15 20:57:43.842736 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.842599    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 15 20:57:43.842736 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.842607    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 15 20:57:43.842736 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.842612    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 15 20:57:43.843793 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.843781    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 15 20:57:43.843793 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.843791    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 15 20:57:43.846357 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.846340    2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-221.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 15 20:57:43.846540 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:43.846508    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-221.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 15 20:57:43.846595 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:43.846562    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 15 20:57:43.847903 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.847886    2564 server.go:1268] "Started kubelet"
Sep 15 20:57:43.848002 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.847965    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 15 20:57:43.848411 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.848280    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 15 20:57:43.848591 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.848572    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 15 20:57:43.851077 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.851029    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 15 20:57:43.852332 ip-10-0-138-221 systemd[1]: Started Kubernetes Kubelet.
Sep 15 20:57:43.854108 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.854091    2564 server.go:310] "Adding debug handlers to kubelet server"
Sep 15 20:57:43.856056 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:43.856033    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 15 20:57:43.857907 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:43.856939    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-221.ec2.internal.18d59a1a1a21287d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-221.ec2.internal,UID:ip-10-0-138-221.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-221.ec2.internal,},FirstTimestamp:2026-09-15 20:57:43.847848061 +0000 UTC m=+0.390010688,LastTimestamp:2026-09-15 20:57:43.847848061 +0000 UTC m=+0.390010688,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-221.ec2.internal,}"
Sep 15 20:57:43.858344 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.858330    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 15 20:57:43.858402 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.858344    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 15 20:57:43.858848 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:43.858450    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-221.ec2.internal\" not found"
Sep 15 20:57:43.859103 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.859086    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 15 20:57:43.859218 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.859201    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 15 20:57:43.859331 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.859172    2564 factory.go:55] Registering systemd factory
Sep 15 20:57:43.859331 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.859275    2564 factory.go:223] Registration of the systemd container factory successfully
Sep 15 20:57:43.859331 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.859299    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 15 20:57:43.859331 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.859333    2564 reconstruct.go:97] "Volume reconstruction finished"
Sep 15 20:57:43.859471 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.859342    2564 reconciler.go:29] "Reconciler: start to sync state"
Sep 15 20:57:43.859524 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.859484    2564 factory.go:153] Registering CRI-O factory
Sep 15 20:57:43.859524 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.859514    2564 factory.go:223] Registration of the crio container factory successfully
Sep 15 20:57:43.859603 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.859559    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 15 20:57:43.859603 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.859581    2564 factory.go:103] Registering Raw factory
Sep 15 20:57:43.859603 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.859595    2564 manager.go:1196] Started watching for new ooms in manager
Sep 15 20:57:43.860014 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.860002    2564 manager.go:349] Starting recovery of all containers
Sep 15 20:57:43.869296 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:43.869266    2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-221.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 15 20:57:43.869296 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:43.869277    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 15 20:57:43.871728 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.871664    2564 manager.go:354] Recovery completed
Sep 15 20:57:43.873192 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.873171    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nw5jg"
Sep 15 20:57:43.876381 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.876370    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:57:43.878209 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.878191    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nw5jg"
Sep 15 20:57:43.878429 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.878414    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-221.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:57:43.878475 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.878464    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-221.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:57:43.878541 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.878483    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-221.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:57:43.879069 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.879047    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 15 20:57:43.879069 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.879062    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 15 20:57:43.879178 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.879079    2564 state_mem.go:36] "Initialized new in-memory state store"
Sep 15 20:57:43.879178 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.879081    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:57:43.880014 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.879998    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-221.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:57:43.880063 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.880023    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-221.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:57:43.880063 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.880033    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-221.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:57:43.881604 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.881592    2564 policy_none.go:49] "None policy: Start"
Sep 15 20:57:43.881652 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.881608    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 15 20:57:43.881652 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.881619    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 15 20:57:43.883838 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.883826    2564 policy_none.go:47] "Start"
Sep 15 20:57:43.942139 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.929226    2564 manager.go:337] "Starting Device Plugin manager"
Sep 15 20:57:43.942139 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:43.929266    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 15 20:57:43.942139 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.929278    2564 server.go:87] "Starting device plugin registration server"
Sep 15 20:57:43.942139 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.929540    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 15 20:57:43.942139 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.929550    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 15 20:57:43.942139 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.929681    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 15 20:57:43.942139 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.929774    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 15 20:57:43.942139 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.929787    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 15 20:57:43.942139 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:43.930321    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 15 20:57:43.942139 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:43.930370    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-221.ec2.internal\" not found"
Sep 15 20:57:43.964435 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.964400    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 15 20:57:43.965860 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.965843    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 15 20:57:43.965910 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.965900    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 15 20:57:43.965954 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.965927    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 15 20:57:43.966001 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:43.965972    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 15 20:57:43.967716 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:43.967697    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 20:57:44.030215 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.030101    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:57:44.031855 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.031830    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-221.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:57:44.031981 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.031875    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-221.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:57:44.031981 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.031891    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-221.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:57:44.031981 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.031920    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-221.ec2.internal"
Sep 15 20:57:44.040238 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.040222    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-221.ec2.internal"
Sep 15 20:57:44.040300 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:44.040248    2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-221.ec2.internal\": node \"ip-10-0-138-221.ec2.internal\" not found"
Sep 15 20:57:44.063860 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:44.063833    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-221.ec2.internal\" not found"
Sep 15 20:57:44.066831 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.066810    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-221.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-221.ec2.internal"]
Sep 15 20:57:44.066889 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.066881    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:57:44.067834 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.067811    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-221.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:57:44.067887 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.067840    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-221.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:57:44.067887 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.067857    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-221.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:57:44.070160 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.070149    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:57:44.070284 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.070258    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-221.ec2.internal"
Sep 15 20:57:44.070327 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.070291    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:57:44.071001 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.070987    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-221.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:57:44.071060 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.071012    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-221.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:57:44.071060 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.071026    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-221.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:57:44.071060 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.071046    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-221.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:57:44.071163 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.071071    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-221.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:57:44.071163 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.071085    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-221.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:57:44.073143 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.073130    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-221.ec2.internal"
Sep 15 20:57:44.073188 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.073154    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 20:57:44.073827 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.073810    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-221.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 20:57:44.073939 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.073838    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-221.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 20:57:44.073939 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.073851    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-221.ec2.internal" event="NodeHasSufficientPID"
Sep 15 20:57:44.097251 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:44.097227    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-221.ec2.internal\" not found" node="ip-10-0-138-221.ec2.internal"
Sep 15 20:57:44.101913 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:44.101895    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-221.ec2.internal\" not found" node="ip-10-0-138-221.ec2.internal"
Sep 15 20:57:44.161349 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.161313    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e947b5ccc9cdb54b46a5b4771d1b37b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-221.ec2.internal\" (UID: \"7e947b5ccc9cdb54b46a5b4771d1b37b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-221.ec2.internal"
Sep 15 20:57:44.161349 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.161346    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6ea81e5eea0bb90932530ee9266ef4ab-config\") pod \"kube-apiserver-proxy-ip-10-0-138-221.ec2.internal\" (UID: \"6ea81e5eea0bb90932530ee9266ef4ab\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-221.ec2.internal"
Sep 15 20:57:44.161557 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.161364    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7e947b5ccc9cdb54b46a5b4771d1b37b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-221.ec2.internal\" (UID: \"7e947b5ccc9cdb54b46a5b4771d1b37b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-221.ec2.internal"
Sep 15 20:57:44.164422 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:44.164407    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-221.ec2.internal\" not found"
Sep 15 20:57:44.261973 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.261928    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e947b5ccc9cdb54b46a5b4771d1b37b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-221.ec2.internal\" (UID: \"7e947b5ccc9cdb54b46a5b4771d1b37b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-221.ec2.internal"
Sep 15 20:57:44.261973 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.261981    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6ea81e5eea0bb90932530ee9266ef4ab-config\") pod \"kube-apiserver-proxy-ip-10-0-138-221.ec2.internal\" (UID: \"6ea81e5eea0bb90932530ee9266ef4ab\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-221.ec2.internal"
Sep 15 20:57:44.262092 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.262029    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6ea81e5eea0bb90932530ee9266ef4ab-config\") pod \"kube-apiserver-proxy-ip-10-0-138-221.ec2.internal\" (UID: \"6ea81e5eea0bb90932530ee9266ef4ab\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-221.ec2.internal"
Sep 15 20:57:44.262092 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.262047    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e947b5ccc9cdb54b46a5b4771d1b37b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-221.ec2.internal\" (UID: \"7e947b5ccc9cdb54b46a5b4771d1b37b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-221.ec2.internal"
Sep 15 20:57:44.262092 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.262072    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7e947b5ccc9cdb54b46a5b4771d1b37b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-221.ec2.internal\" (UID: \"7e947b5ccc9cdb54b46a5b4771d1b37b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-221.ec2.internal"
Sep 15 20:57:44.262178 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.262097    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7e947b5ccc9cdb54b46a5b4771d1b37b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-221.ec2.internal\" (UID: \"7e947b5ccc9cdb54b46a5b4771d1b37b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-221.ec2.internal"
Sep 15 20:57:44.265035 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:44.265013    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-221.ec2.internal\" not found"
Sep 15 20:57:44.365886 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:44.365820    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-221.ec2.internal\" not found"
Sep 15 20:57:44.400007 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.399981    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-221.ec2.internal"
Sep 15 20:57:44.405195 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.404874    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-221.ec2.internal"
Sep 15 20:57:44.466938 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:44.466901    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-221.ec2.internal\" not found"
Sep 15 20:57:44.567470 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:44.567437    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-221.ec2.internal\" not found"
Sep 15 20:57:44.668097 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:44.668022    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-221.ec2.internal\" not found"
Sep 15 20:57:44.768650 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:44.768618    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-221.ec2.internal\" not found"
Sep 15 20:57:44.792815 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.792784    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 15 20:57:44.792957 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.792939    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 15 20:57:44.843710 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.843684    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 20:57:44.858909 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.858878    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-221.ec2.internal"
Sep 15 20:57:44.859051 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.858883    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 15 20:57:44.870800 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.870772    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 15 20:57:44.872725 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.872708    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 15 20:57:44.874590 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.874575    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-221.ec2.internal"
Sep 15 20:57:44.880801 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.880782    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 15 20:57:44.883971 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.883947    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-14 20:52:43 +0000 UTC" deadline="2028-03-02 06:00:23.69074918 +0000 UTC"
Sep 15 20:57:44.884018 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.883972    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12801h2m38.806780108s"
Sep 15 20:57:44.891441 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.891427    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6jdcc"
Sep 15 20:57:44.897036 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.897017    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6jdcc"
Sep 15 20:57:44.980309 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:44.980231    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 20:57:45.084868 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.084829    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 20:57:45.163223 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:45.162974    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ea81e5eea0bb90932530ee9266ef4ab.slice/crio-a61d835c8600e10d8e28185ca4822b723f2fa9b7cbdc09234ce8a21dc2c84b4f WatchSource:0}: Error finding container a61d835c8600e10d8e28185ca4822b723f2fa9b7cbdc09234ce8a21dc2c84b4f: Status 404 returned error can't find the container with id a61d835c8600e10d8e28185ca4822b723f2fa9b7cbdc09234ce8a21dc2c84b4f
Sep 15 20:57:45.168203 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.168187    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 20:57:45.298586 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:45.298557    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e947b5ccc9cdb54b46a5b4771d1b37b.slice/crio-a9c0b0266c85b3e7f9114bbf330312e6251f315a243516859d46e1ee1cf8b84f WatchSource:0}: Error finding container a9c0b0266c85b3e7f9114bbf330312e6251f315a243516859d46e1ee1cf8b84f: Status 404 returned error can't find the container with id a9c0b0266c85b3e7f9114bbf330312e6251f315a243516859d46e1ee1cf8b84f
Sep 15 20:57:45.803406 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.803379    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 20:57:45.835837 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.835803    2564 apiserver.go:52] "Watching apiserver"
Sep 15 20:57:45.847951 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.847918    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 15 20:57:45.848395 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.848359    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-gwqhk","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg","openshift-dns/node-resolver-mt7b6","openshift-image-registry/node-ca-cdrtr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-221.ec2.internal","openshift-multus/network-metrics-daemon-4dws2","openshift-network-diagnostics/network-check-target-m8fpw","openshift-network-operator/iptables-alerter-h94k8","kube-system/global-pull-secret-syncer-pjz7r","kube-system/kube-apiserver-proxy-ip-10-0-138-221.ec2.internal","openshift-cluster-node-tuning-operator/tuned-m94mk","openshift-multus/multus-additional-cni-plugins-b8vrp","openshift-multus/multus-cq8j7","openshift-ovn-kubernetes/ovnkube-node-d4c7k"]
Sep 15 20:57:45.851102 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.851072    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gwqhk"
Sep 15 20:57:45.853850 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.853512    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg"
Sep 15 20:57:45.854337 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.854311    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 15 20:57:45.854435 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.854351    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-pnmqq\""
Sep 15 20:57:45.854435 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.854409    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 15 20:57:45.855794 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.855776    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mt7b6"
Sep 15 20:57:45.856174 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.856157    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 15 20:57:45.856473 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.856457    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 15 20:57:45.856855 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.856834    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-z5b9k\""
Sep 15 20:57:45.856976 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.856861    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 15 20:57:45.858202 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.858184    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pxpd8\""
Sep 15 20:57:45.858503 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.858484    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 15 20:57:45.858730 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.858713    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 15 20:57:45.859051 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.859034    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cdrtr"
Sep 15 20:57:45.860808 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.860688    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:57:45.860808 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:45.860755    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dws2" podUID="ce801675-8966-401c-b202-6631f49bdea0"
Sep 15 20:57:45.861336 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.861319    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 15 20:57:45.861639 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.861624    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 15 20:57:45.861876 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.861832    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zxzzq\""
Sep 15 20:57:45.861960 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.861888    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 15 20:57:45.863238 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.863216    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:57:45.863334 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:45.863281    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m8fpw" podUID="a70ba96c-ccb0-436a-aa3c-e056cb9a76e3"
Sep 15 20:57:45.865589 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.865569    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h94k8"
Sep 15 20:57:45.868056 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.867946    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:57:45.868056 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:45.868017    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjz7r" podUID="88840482-9724-4d2b-8343-3ed2ad33b258"
Sep 15 20:57:45.868214 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.868188    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 15 20:57:45.868534 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.868484    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7xrrg\""
Sep 15 20:57:45.868619 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.868581    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 15 20:57:45.868939 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.868754    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 15 20:57:45.871085 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.870581    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:45.871085 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.870647    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs\") pod \"network-metrics-daemon-4dws2\" (UID: \"ce801675-8966-401c-b202-6631f49bdea0\") " pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:57:45.871085 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.871049    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bc773606-78e0-4ee0-a8ac-29d191e7f936-agent-certs\") pod \"konnectivity-agent-gwqhk\" (UID: \"bc773606-78e0-4ee0-a8ac-29d191e7f936\") " pod="kube-system/konnectivity-agent-gwqhk"
Sep 15 20:57:45.871085 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.871079    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d8xh\" (UniqueName: \"kubernetes.io/projected/0bf2384a-bc40-40a7-a139-98a53ebf6b68-kube-api-access-5d8xh\") pod \"aws-ebs-csi-driver-node-x87mg\" (UID: \"0bf2384a-bc40-40a7-a139-98a53ebf6b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg"
Sep 15 20:57:45.871340 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.871111    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7c4d34d1-b3a5-4974-9db9-0ccf8786db47-hosts-file\") pod \"node-resolver-mt7b6\" (UID: \"7c4d34d1-b3a5-4974-9db9-0ccf8786db47\") " pod="openshift-dns/node-resolver-mt7b6"
Sep 15 20:57:45.871340 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.871134    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c4d34d1-b3a5-4974-9db9-0ccf8786db47-tmp-dir\") pod \"node-resolver-mt7b6\" (UID: \"7c4d34d1-b3a5-4974-9db9-0ccf8786db47\") " pod="openshift-dns/node-resolver-mt7b6"
Sep 15 20:57:45.871340 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.871216    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/afadf0fd-9154-4a32-8abe-d8b29dfcaa90-serviceca\") pod \"node-ca-cdrtr\" (UID: \"afadf0fd-9154-4a32-8abe-d8b29dfcaa90\") " pod="openshift-image-registry/node-ca-cdrtr"
Sep 15 20:57:45.871340 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.871242    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bc773606-78e0-4ee0-a8ac-29d191e7f936-konnectivity-ca\") pod \"konnectivity-agent-gwqhk\" (UID: \"bc773606-78e0-4ee0-a8ac-29d191e7f936\") " pod="kube-system/konnectivity-agent-gwqhk"
Sep 15 20:57:45.871340 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.871289    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0bf2384a-bc40-40a7-a139-98a53ebf6b68-socket-dir\") pod \"aws-ebs-csi-driver-node-x87mg\" (UID: \"0bf2384a-bc40-40a7-a139-98a53ebf6b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg"
Sep 15 20:57:45.871610 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.871352    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6mqw\" (UniqueName: \"kubernetes.io/projected/7c4d34d1-b3a5-4974-9db9-0ccf8786db47-kube-api-access-d6mqw\") pod \"node-resolver-mt7b6\" (UID: \"7c4d34d1-b3a5-4974-9db9-0ccf8786db47\") " pod="openshift-dns/node-resolver-mt7b6"
Sep 15 20:57:45.871610 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.871378    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0bf2384a-bc40-40a7-a139-98a53ebf6b68-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x87mg\" (UID: \"0bf2384a-bc40-40a7-a139-98a53ebf6b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg"
Sep 15 20:57:45.871610 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.871415    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0bf2384a-bc40-40a7-a139-98a53ebf6b68-registration-dir\") pod \"aws-ebs-csi-driver-node-x87mg\" (UID: \"0bf2384a-bc40-40a7-a139-98a53ebf6b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg"
Sep 15 20:57:45.871610 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.871454    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0bf2384a-bc40-40a7-a139-98a53ebf6b68-device-dir\") pod \"aws-ebs-csi-driver-node-x87mg\" (UID: \"0bf2384a-bc40-40a7-a139-98a53ebf6b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg"
Sep 15 20:57:45.871610 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.871519    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0bf2384a-bc40-40a7-a139-98a53ebf6b68-etc-selinux\") pod \"aws-ebs-csi-driver-node-x87mg\" (UID: \"0bf2384a-bc40-40a7-a139-98a53ebf6b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg"
Sep 15 20:57:45.871610 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.871555    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/afadf0fd-9154-4a32-8abe-d8b29dfcaa90-host\") pod \"node-ca-cdrtr\" (UID: \"afadf0fd-9154-4a32-8abe-d8b29dfcaa90\") " pod="openshift-image-registry/node-ca-cdrtr"
Sep 15 20:57:45.871610 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.871582    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx95w\" (UniqueName: \"kubernetes.io/projected/afadf0fd-9154-4a32-8abe-d8b29dfcaa90-kube-api-access-dx95w\") pod \"node-ca-cdrtr\" (UID: \"afadf0fd-9154-4a32-8abe-d8b29dfcaa90\") " pod="openshift-image-registry/node-ca-cdrtr"
Sep 15 20:57:45.871915 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.871612    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hglmr\" (UniqueName: \"kubernetes.io/projected/ce801675-8966-401c-b202-6631f49bdea0-kube-api-access-hglmr\") pod \"network-metrics-daemon-4dws2\" (UID: \"ce801675-8966-401c-b202-6631f49bdea0\") " pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:57:45.871915 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.871637    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0bf2384a-bc40-40a7-a139-98a53ebf6b68-sys-fs\") pod \"aws-ebs-csi-driver-node-x87mg\" (UID: \"0bf2384a-bc40-40a7-a139-98a53ebf6b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg"
Sep 15 20:57:45.873824 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.873219    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b8vrp"
Sep 15 20:57:45.873824 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.873426    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 15 20:57:45.873824 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.873700    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4ng2n\""
Sep 15 20:57:45.873824 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.873759    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 15 20:57:45.876183 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.876165    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:45.876536 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.876494    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 15 20:57:45.876852 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.876837    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 15 20:57:45.876962 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.876945    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sct9r\""
Sep 15 20:57:45.877204 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.877184    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 15 20:57:45.877298 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.877277    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 15 20:57:45.877353 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.877333    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 15 20:57:45.878749 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.878730    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:45.879102 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.878975    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 15 20:57:45.879802 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.879780    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4x6zs\""
Sep 15 20:57:45.881224 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.881206    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 15 20:57:45.882705 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.882684    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 15 20:57:45.882788 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.882707    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 15 20:57:45.882979 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.882962    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 15 20:57:45.885842 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.883796    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8cgfm\""
Sep 15 20:57:45.885842 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.883896    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 15 20:57:45.885842 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.883952    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 15 20:57:45.901666 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.901595    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 20:52:44 +0000 UTC" deadline="2028-04-27 02:59:34.76453262 +0000 UTC"
Sep 15 20:57:45.901666 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.901666    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14142h1m48.86287188s"
Sep 15 20:57:45.961050 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.961016    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 15 20:57:45.971920 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.971889    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-host-run-netns\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:45.972091 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.971934    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-run-ovn\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:45.972091 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.971964    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fc205950-caf0-4c47-84ce-a8fc54a1afb2-env-overrides\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:45.972091 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.971996    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6mqw\" (UniqueName: \"kubernetes.io/projected/7c4d34d1-b3a5-4974-9db9-0ccf8786db47-kube-api-access-d6mqw\") pod \"node-resolver-mt7b6\" (UID: \"7c4d34d1-b3a5-4974-9db9-0ccf8786db47\") " pod="openshift-dns/node-resolver-mt7b6"
Sep 15 20:57:45.972091 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972023    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-etc-sysctl-d\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:45.972091 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972058    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-tmp\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:45.972091 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972085    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b24f6335-59f4-4b7a-8548-56465ed60966-cni-binary-copy\") pod \"multus-additional-cni-plugins-b8vrp\" (UID: \"b24f6335-59f4-4b7a-8548-56465ed60966\") " pod="openshift-multus/multus-additional-cni-plugins-b8vrp"
Sep 15 20:57:45.972393 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972113    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb7pb\" (UniqueName: \"kubernetes.io/projected/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-kube-api-access-gb7pb\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:45.972393 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972139    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-run\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:45.972393 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972162    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-var-lib-kubelet\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:45.972393 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972197    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-cnibin\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:45.972393 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972220    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-cni-binary-copy\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:45.972393 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972258    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hglmr\" (UniqueName: \"kubernetes.io/projected/ce801675-8966-401c-b202-6631f49bdea0-kube-api-access-hglmr\") pod \"network-metrics-daemon-4dws2\" (UID: \"ce801675-8966-401c-b202-6631f49bdea0\") " pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:57:45.972393 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972291    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvqjl\" (UniqueName: \"kubernetes.io/projected/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-kube-api-access-mvqjl\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:45.972393 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972318    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-multus-daemon-config\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:45.972393 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972344    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fc205950-caf0-4c47-84ce-a8fc54a1afb2-ovnkube-config\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:45.972393 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972369    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c4d34d1-b3a5-4974-9db9-0ccf8786db47-tmp-dir\") pod \"node-resolver-mt7b6\" (UID: \"7c4d34d1-b3a5-4974-9db9-0ccf8786db47\") " pod="openshift-dns/node-resolver-mt7b6"
Sep 15 20:57:45.972393 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972396    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-host-run-ovn-kubernetes\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:45.972975 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972421    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fc205950-caf0-4c47-84ce-a8fc54a1afb2-ovn-node-metrics-cert\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:45.972975 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972447    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bc773606-78e0-4ee0-a8ac-29d191e7f936-konnectivity-ca\") pod \"konnectivity-agent-gwqhk\" (UID: \"bc773606-78e0-4ee0-a8ac-29d191e7f936\") " pod="kube-system/konnectivity-agent-gwqhk"
Sep 15 20:57:45.972975 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972474    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqfhs\" (UniqueName: \"kubernetes.io/projected/928ab144-6fdf-4555-abdf-86b1d0dc8a72-kube-api-access-pqfhs\") pod \"iptables-alerter-h94k8\" (UID: \"928ab144-6fdf-4555-abdf-86b1d0dc8a72\") " pod="openshift-network-operator/iptables-alerter-h94k8"
Sep 15 20:57:45.972975 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972514    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/88840482-9724-4d2b-8343-3ed2ad33b258-kubelet-config\") pod \"global-pull-secret-syncer-pjz7r\" (UID: \"88840482-9724-4d2b-8343-3ed2ad33b258\") " pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:57:45.972975 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972561    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-lib-modules\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:45.972975 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972586    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-var-lib-openvswitch\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:45.972975 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972612    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-multus-cni-dir\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:45.972975 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972652    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0bf2384a-bc40-40a7-a139-98a53ebf6b68-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x87mg\" (UID: \"0bf2384a-bc40-40a7-a139-98a53ebf6b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg"
Sep 15 20:57:45.972975 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972678    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0bf2384a-bc40-40a7-a139-98a53ebf6b68-etc-selinux\") pod \"aws-ebs-csi-driver-node-x87mg\" (UID: \"0bf2384a-bc40-40a7-a139-98a53ebf6b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg"
Sep 15 20:57:45.972975 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972702    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/afadf0fd-9154-4a32-8abe-d8b29dfcaa90-host\") pod \"node-ca-cdrtr\" (UID: \"afadf0fd-9154-4a32-8abe-d8b29dfcaa90\") " pod="openshift-image-registry/node-ca-cdrtr"
Sep 15 20:57:45.972975 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972725    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-etc-kubernetes\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:45.972975 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972750    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-hostroot\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:45.972975 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972779    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-multus-conf-dir\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:45.972975 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972825    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-host-kubelet\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:45.972975 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972862    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0bf2384a-bc40-40a7-a139-98a53ebf6b68-sys-fs\") pod \"aws-ebs-csi-driver-node-x87mg\" (UID: \"0bf2384a-bc40-40a7-a139-98a53ebf6b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg"
Sep 15 20:57:45.972975 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972881    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkkcq\" (UniqueName: \"kubernetes.io/projected/b24f6335-59f4-4b7a-8548-56465ed60966-kube-api-access-qkkcq\") pod \"multus-additional-cni-plugins-b8vrp\" (UID: \"b24f6335-59f4-4b7a-8548-56465ed60966\") " pod="openshift-multus/multus-additional-cni-plugins-b8vrp"
Sep 15 20:57:45.972975 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972907    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-host-run-multus-certs\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:45.973659 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972929    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-systemd-units\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:45.973659 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972954    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-run-systemd\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:45.973659 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972975    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-etc-openvswitch\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:45.973659 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.972991    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-host-cni-netd\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:45.973659 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973025    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:45.973659 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973057    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs\") pod \"network-metrics-daemon-4dws2\" (UID: \"ce801675-8966-401c-b202-6631f49bdea0\") " pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:57:45.973659 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973106    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d8xh\" (UniqueName: \"kubernetes.io/projected/0bf2384a-bc40-40a7-a139-98a53ebf6b68-kube-api-access-5d8xh\") pod \"aws-ebs-csi-driver-node-x87mg\" (UID: \"0bf2384a-bc40-40a7-a139-98a53ebf6b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg"
Sep 15 20:57:45.973659 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973138    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/928ab144-6fdf-4555-abdf-86b1d0dc8a72-host-slash\") pod \"iptables-alerter-h94k8\" (UID: \"928ab144-6fdf-4555-abdf-86b1d0dc8a72\") " pod="openshift-network-operator/iptables-alerter-h94k8"
Sep 15 20:57:45.973659 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973189    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-host-run-k8s-cni-cncf-io\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:45.973659 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973250    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-etc-kubernetes\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:45.973659 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973277    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-node-log\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:45.973659 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973293    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-system-cni-dir\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:45.973659 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973310    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-host-var-lib-cni-multus\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:45.973659 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973324    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fc205950-caf0-4c47-84ce-a8fc54a1afb2-ovnkube-script-lib\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:45.973659 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973339    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt7pz\" (UniqueName: \"kubernetes.io/projected/fc205950-caf0-4c47-84ce-a8fc54a1afb2-kube-api-access-gt7pz\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:45.973659 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973365    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0bf2384a-bc40-40a7-a139-98a53ebf6b68-device-dir\") pod \"aws-ebs-csi-driver-node-x87mg\" (UID: \"0bf2384a-bc40-40a7-a139-98a53ebf6b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg"
Sep 15 20:57:45.974224 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973404    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-host-var-lib-kubelet\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:45.974224 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973498    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bc773606-78e0-4ee0-a8ac-29d191e7f936-konnectivity-ca\") pod \"konnectivity-agent-gwqhk\" (UID: \"bc773606-78e0-4ee0-a8ac-29d191e7f936\") " pod="kube-system/konnectivity-agent-gwqhk"
Sep 15 20:57:45.974224 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973509    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0bf2384a-bc40-40a7-a139-98a53ebf6b68-sys-fs\") pod \"aws-ebs-csi-driver-node-x87mg\" (UID: \"0bf2384a-bc40-40a7-a139-98a53ebf6b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg"
Sep 15 20:57:45.974224 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973500    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0bf2384a-bc40-40a7-a139-98a53ebf6b68-device-dir\") pod \"aws-ebs-csi-driver-node-x87mg\" (UID: \"0bf2384a-bc40-40a7-a139-98a53ebf6b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg"
Sep 15 20:57:45.974224 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973597    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0bf2384a-bc40-40a7-a139-98a53ebf6b68-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x87mg\" (UID: \"0bf2384a-bc40-40a7-a139-98a53ebf6b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg"
Sep 15 20:57:45.974224 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973646    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-etc-sysconfig\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:45.974224 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973673    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0bf2384a-bc40-40a7-a139-98a53ebf6b68-etc-selinux\") pod \"aws-ebs-csi-driver-node-x87mg\" (UID: \"0bf2384a-bc40-40a7-a139-98a53ebf6b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg"
Sep 15 20:57:45.974224 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973679    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcpx9\" (UniqueName: \"kubernetes.io/projected/a70ba96c-ccb0-436a-aa3c-e056cb9a76e3-kube-api-access-dcpx9\") pod \"network-check-target-m8fpw\" (UID: \"a70ba96c-ccb0-436a-aa3c-e056cb9a76e3\") " pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:57:45.974224 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973710    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-host-var-lib-cni-bin\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:45.974224 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973722    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c4d34d1-b3a5-4974-9db9-0ccf8786db47-tmp-dir\") pod \"node-resolver-mt7b6\" (UID: \"7c4d34d1-b3a5-4974-9db9-0ccf8786db47\") " pod="openshift-dns/node-resolver-mt7b6"
Sep 15 20:57:45.974224 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973736    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-host-cni-bin\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:45.974224 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973768    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bc773606-78e0-4ee0-a8ac-29d191e7f936-agent-certs\") pod \"konnectivity-agent-gwqhk\" (UID: \"bc773606-78e0-4ee0-a8ac-29d191e7f936\") " pod="kube-system/konnectivity-agent-gwqhk"
Sep 15 20:57:45.974224 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:45.973771    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:57:45.974224 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973821    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/afadf0fd-9154-4a32-8abe-d8b29dfcaa90-serviceca\") pod \"node-ca-cdrtr\" (UID: \"afadf0fd-9154-4a32-8abe-d8b29dfcaa90\") " pod="openshift-image-registry/node-ca-cdrtr"
Sep 15 20:57:45.974224 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973854    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88840482-9724-4d2b-8343-3ed2ad33b258-original-pull-secret\") pod \"global-pull-secret-syncer-pjz7r\" (UID: \"88840482-9724-4d2b-8343-3ed2ad33b258\") " pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:57:45.974224 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973891    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-host\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:45.974224 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973920    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-etc-tuned\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:45.975063 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:45.973948    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs podName:ce801675-8966-401c-b202-6631f49bdea0 nodeName:}" failed. No retries permitted until 2026-09-15 20:57:46.473927963 +0000 UTC m=+3.016090576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs") pod "network-metrics-daemon-4dws2" (UID: "ce801675-8966-401c-b202-6631f49bdea0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:57:45.975063 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.973997    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/afadf0fd-9154-4a32-8abe-d8b29dfcaa90-host\") pod \"node-ca-cdrtr\" (UID: \"afadf0fd-9154-4a32-8abe-d8b29dfcaa90\") " pod="openshift-image-registry/node-ca-cdrtr"
Sep 15 20:57:45.975063 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974039    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-run-openvswitch\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:45.975063 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974062    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-log-socket\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:45.975063 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974080    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b24f6335-59f4-4b7a-8548-56465ed60966-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b8vrp\" (UID: \"b24f6335-59f4-4b7a-8548-56465ed60966\") " pod="openshift-multus/multus-additional-cni-plugins-b8vrp"
Sep 15 20:57:45.975063 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974111    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0bf2384a-bc40-40a7-a139-98a53ebf6b68-socket-dir\") pod \"aws-ebs-csi-driver-node-x87mg\" (UID: \"0bf2384a-bc40-40a7-a139-98a53ebf6b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg"
Sep 15 20:57:45.975063 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974139    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b24f6335-59f4-4b7a-8548-56465ed60966-system-cni-dir\") pod \"multus-additional-cni-plugins-b8vrp\" (UID: \"b24f6335-59f4-4b7a-8548-56465ed60966\") " pod="openshift-multus/multus-additional-cni-plugins-b8vrp"
Sep 15 20:57:45.975063 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974153    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b24f6335-59f4-4b7a-8548-56465ed60966-cnibin\") pod \"multus-additional-cni-plugins-b8vrp\" (UID: \"b24f6335-59f4-4b7a-8548-56465ed60966\") " pod="openshift-multus/multus-additional-cni-plugins-b8vrp"
Sep 15 20:57:45.975063 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974172    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b24f6335-59f4-4b7a-8548-56465ed60966-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b8vrp\" (UID: \"b24f6335-59f4-4b7a-8548-56465ed60966\") " pod="openshift-multus/multus-additional-cni-plugins-b8vrp"
Sep 15 20:57:45.975063 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974201    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0bf2384a-bc40-40a7-a139-98a53ebf6b68-registration-dir\") pod \"aws-ebs-csi-driver-node-x87mg\" (UID: \"0bf2384a-bc40-40a7-a139-98a53ebf6b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg"
Sep 15 20:57:45.975063 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974240    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx95w\" (UniqueName: \"kubernetes.io/projected/afadf0fd-9154-4a32-8abe-d8b29dfcaa90-kube-api-access-dx95w\") pod \"node-ca-cdrtr\" (UID: \"afadf0fd-9154-4a32-8abe-d8b29dfcaa90\") " pod="openshift-image-registry/node-ca-cdrtr"
Sep 15 20:57:45.975063 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974264    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-etc-modprobe-d\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:45.975063 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974276    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/afadf0fd-9154-4a32-8abe-d8b29dfcaa90-serviceca\") pod \"node-ca-cdrtr\" (UID: \"afadf0fd-9154-4a32-8abe-d8b29dfcaa90\") " pod="openshift-image-registry/node-ca-cdrtr"
Sep 15 20:57:45.975063 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974285    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-sys\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:45.975063 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974300    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/928ab144-6fdf-4555-abdf-86b1d0dc8a72-iptables-alerter-script\") pod \"iptables-alerter-h94k8\" (UID: \"928ab144-6fdf-4555-abdf-86b1d0dc8a72\") " pod="openshift-network-operator/iptables-alerter-h94k8"
Sep 15 20:57:45.975063 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974316    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/88840482-9724-4d2b-8343-3ed2ad33b258-dbus\") pod \"global-pull-secret-syncer-pjz7r\" (UID: \"88840482-9724-4d2b-8343-3ed2ad33b258\") " pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:57:45.975869 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974436    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0bf2384a-bc40-40a7-a139-98a53ebf6b68-socket-dir\") pod \"aws-ebs-csi-driver-node-x87mg\" (UID: \"0bf2384a-bc40-40a7-a139-98a53ebf6b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg"
Sep 15 20:57:45.975869 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974519    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 15 20:57:45.975869 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974699    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0bf2384a-bc40-40a7-a139-98a53ebf6b68-registration-dir\") pod \"aws-ebs-csi-driver-node-x87mg\" (UID: \"0bf2384a-bc40-40a7-a139-98a53ebf6b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg"
Sep 15 20:57:45.975869 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974737    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-multus-socket-dir-parent\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:45.975869 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974762    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-host-run-netns\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:45.975869 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974787    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7c4d34d1-b3a5-4974-9db9-0ccf8786db47-hosts-file\") pod \"node-resolver-mt7b6\" (UID: \"7c4d34d1-b3a5-4974-9db9-0ccf8786db47\") " pod="openshift-dns/node-resolver-mt7b6"
Sep 15 20:57:45.975869 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974824    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-etc-sysctl-conf\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:45.975869 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974913    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-etc-systemd\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:45.975869 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974939    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b24f6335-59f4-4b7a-8548-56465ed60966-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b8vrp\" (UID: \"b24f6335-59f4-4b7a-8548-56465ed60966\") " pod="openshift-multus/multus-additional-cni-plugins-b8vrp"
Sep 15 20:57:45.975869 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.974965    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-host-slash\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:45.975869 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.975025    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7c4d34d1-b3a5-4974-9db9-0ccf8786db47-hosts-file\") pod \"node-resolver-mt7b6\" (UID: \"7c4d34d1-b3a5-4974-9db9-0ccf8786db47\") " pod="openshift-dns/node-resolver-mt7b6"
Sep 15 20:57:45.976458 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.976236    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-221.ec2.internal" event={"ID":"7e947b5ccc9cdb54b46a5b4771d1b37b","Type":"ContainerStarted","Data":"a9c0b0266c85b3e7f9114bbf330312e6251f315a243516859d46e1ee1cf8b84f"}
Sep 15 20:57:45.980076 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.980046    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bc773606-78e0-4ee0-a8ac-29d191e7f936-agent-certs\") pod \"konnectivity-agent-gwqhk\" (UID: \"bc773606-78e0-4ee0-a8ac-29d191e7f936\") " pod="kube-system/konnectivity-agent-gwqhk"
Sep 15 20:57:45.980208 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.980189    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-221.ec2.internal" event={"ID":"6ea81e5eea0bb90932530ee9266ef4ab","Type":"ContainerStarted","Data":"a61d835c8600e10d8e28185ca4822b723f2fa9b7cbdc09234ce8a21dc2c84b4f"}
Sep 15 20:57:45.986145 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.986115    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6mqw\" (UniqueName: \"kubernetes.io/projected/7c4d34d1-b3a5-4974-9db9-0ccf8786db47-kube-api-access-d6mqw\") pod \"node-resolver-mt7b6\" (UID: \"7c4d34d1-b3a5-4974-9db9-0ccf8786db47\") " pod="openshift-dns/node-resolver-mt7b6"
Sep 15 20:57:45.989389 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.988975    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d8xh\" (UniqueName: \"kubernetes.io/projected/0bf2384a-bc40-40a7-a139-98a53ebf6b68-kube-api-access-5d8xh\") pod \"aws-ebs-csi-driver-node-x87mg\" (UID: \"0bf2384a-bc40-40a7-a139-98a53ebf6b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg"
Sep 15 20:57:45.989389 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.989330    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hglmr\" (UniqueName: \"kubernetes.io/projected/ce801675-8966-401c-b202-6631f49bdea0-kube-api-access-hglmr\") pod \"network-metrics-daemon-4dws2\" (UID: \"ce801675-8966-401c-b202-6631f49bdea0\") " pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:57:45.991004 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:45.990977    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx95w\" (UniqueName: \"kubernetes.io/projected/afadf0fd-9154-4a32-8abe-d8b29dfcaa90-kube-api-access-dx95w\") pod \"node-ca-cdrtr\" (UID: \"afadf0fd-9154-4a32-8abe-d8b29dfcaa90\") " pod="openshift-image-registry/node-ca-cdrtr"
Sep 15 20:57:46.075798 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.075698    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-host-run-netns\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.075798 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.075749    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-etc-sysctl-conf\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.075798 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.075794    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-etc-systemd\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.076033 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.075825    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b24f6335-59f4-4b7a-8548-56465ed60966-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b8vrp\" (UID: \"b24f6335-59f4-4b7a-8548-56465ed60966\") " pod="openshift-multus/multus-additional-cni-plugins-b8vrp"
Sep 15 20:57:46.076033 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.075861    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-host-slash\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.076033 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.075885    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-host-run-netns\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.076033 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.075908    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-run-ovn\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.076033 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.075933    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fc205950-caf0-4c47-84ce-a8fc54a1afb2-env-overrides\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.076033 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.075959    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-etc-sysctl-d\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.076033 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.075981    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-tmp\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.076033 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.076005    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b24f6335-59f4-4b7a-8548-56465ed60966-cni-binary-copy\") pod \"multus-additional-cni-plugins-b8vrp\" (UID: \"b24f6335-59f4-4b7a-8548-56465ed60966\") " pod="openshift-multus/multus-additional-cni-plugins-b8vrp"
Sep 15 20:57:46.076475 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.076034    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb7pb\" (UniqueName: \"kubernetes.io/projected/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-kube-api-access-gb7pb\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.076475 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.076094    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-run\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.076475 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.076116    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-var-lib-kubelet\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.076475 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.076157    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-cnibin\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.076475 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.076179    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-cni-binary-copy\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.076475 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.076201    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvqjl\" (UniqueName: \"kubernetes.io/projected/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-kube-api-access-mvqjl\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.076475 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.076216    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-multus-daemon-config\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.076475 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.076230    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fc205950-caf0-4c47-84ce-a8fc54a1afb2-ovnkube-config\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.076475 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.076252    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-host-run-ovn-kubernetes\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.076475 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.076276    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fc205950-caf0-4c47-84ce-a8fc54a1afb2-ovn-node-metrics-cert\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.076475 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.076302    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqfhs\" (UniqueName: \"kubernetes.io/projected/928ab144-6fdf-4555-abdf-86b1d0dc8a72-kube-api-access-pqfhs\") pod \"iptables-alerter-h94k8\" (UID: \"928ab144-6fdf-4555-abdf-86b1d0dc8a72\") " pod="openshift-network-operator/iptables-alerter-h94k8"
Sep 15 20:57:46.076475 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.076339    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/88840482-9724-4d2b-8343-3ed2ad33b258-kubelet-config\") pod \"global-pull-secret-syncer-pjz7r\" (UID: \"88840482-9724-4d2b-8343-3ed2ad33b258\") " pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:57:46.076475 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.076361    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-lib-modules\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.077137 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.076712    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-var-lib-openvswitch\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.077137 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.076763    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-multus-cni-dir\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.077137 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.076807    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-etc-kubernetes\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.077137 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.076896    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-hostroot\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.077137 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.076918    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-multus-conf-dir\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.077137 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.076937    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-host-kubelet\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.077137 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.076976    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkkcq\" (UniqueName: \"kubernetes.io/projected/b24f6335-59f4-4b7a-8548-56465ed60966-kube-api-access-qkkcq\") pod \"multus-additional-cni-plugins-b8vrp\" (UID: \"b24f6335-59f4-4b7a-8548-56465ed60966\") " pod="openshift-multus/multus-additional-cni-plugins-b8vrp"
Sep 15 20:57:46.077137 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.077010    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-host-run-multus-certs\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.077137 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.077053    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-systemd-units\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.077137 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.077076    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-run\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.077137 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.077099    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-run-systemd\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.077629 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.077145    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-etc-openvswitch\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.077629 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.077177    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-host-cni-netd\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.077629 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.077186    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-var-lib-kubelet\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.077629 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.077217    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.077629 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.077254    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-cnibin\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.077629 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.077275    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/928ab144-6fdf-4555-abdf-86b1d0dc8a72-host-slash\") pod \"iptables-alerter-h94k8\" (UID: \"928ab144-6fdf-4555-abdf-86b1d0dc8a72\") " pod="openshift-network-operator/iptables-alerter-h94k8"
Sep 15 20:57:46.077896 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.077844    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-cni-binary-copy\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.078610 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.078573    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-host-run-k8s-cni-cncf-io\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.078724 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.078638    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-etc-kubernetes\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.078724 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.078664    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-lib-modules\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.078724 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.078666    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-node-log\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.078872 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.078730    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-system-cni-dir\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.078872 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.078733    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/928ab144-6fdf-4555-abdf-86b1d0dc8a72-host-slash\") pod \"iptables-alerter-h94k8\" (UID: \"928ab144-6fdf-4555-abdf-86b1d0dc8a72\") " pod="openshift-network-operator/iptables-alerter-h94k8"
Sep 15 20:57:46.078872 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.078764    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-host-var-lib-cni-multus\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.078872 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.078790    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-host-run-netns\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.078872 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.078813    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fc205950-caf0-4c47-84ce-a8fc54a1afb2-ovnkube-script-lib\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.078872 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.078856    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt7pz\" (UniqueName: \"kubernetes.io/projected/fc205950-caf0-4c47-84ce-a8fc54a1afb2-kube-api-access-gt7pz\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.079074 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.078881    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-host-var-lib-kubelet\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.079074 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.078908    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-etc-sysconfig\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.079074 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.078940    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcpx9\" (UniqueName: \"kubernetes.io/projected/a70ba96c-ccb0-436a-aa3c-e056cb9a76e3-kube-api-access-dcpx9\") pod \"network-check-target-m8fpw\" (UID: \"a70ba96c-ccb0-436a-aa3c-e056cb9a76e3\") " pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:57:46.079074 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.078992    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-etc-systemd\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.079074 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.078998    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-host-var-lib-cni-bin\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.079219 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.079058    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-host-cni-bin\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.079219 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.079107    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88840482-9724-4d2b-8343-3ed2ad33b258-original-pull-secret\") pod \"global-pull-secret-syncer-pjz7r\" (UID: \"88840482-9724-4d2b-8343-3ed2ad33b258\") " pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:57:46.079219 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.079140    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-host\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.079219 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.079168    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-etc-tuned\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.079331 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.079233    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-run-openvswitch\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.079331 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.079285    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-log-socket\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.079430 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.079360    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b24f6335-59f4-4b7a-8548-56465ed60966-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b8vrp\" (UID: \"b24f6335-59f4-4b7a-8548-56465ed60966\") " pod="openshift-multus/multus-additional-cni-plugins-b8vrp"
Sep 15 20:57:46.079430 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.079398    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b24f6335-59f4-4b7a-8548-56465ed60966-system-cni-dir\") pod \"multus-additional-cni-plugins-b8vrp\" (UID: \"b24f6335-59f4-4b7a-8548-56465ed60966\") " pod="openshift-multus/multus-additional-cni-plugins-b8vrp"
Sep 15 20:57:46.079492 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.079446    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b24f6335-59f4-4b7a-8548-56465ed60966-cnibin\") pod \"multus-additional-cni-plugins-b8vrp\" (UID: \"b24f6335-59f4-4b7a-8548-56465ed60966\") " pod="openshift-multus/multus-additional-cni-plugins-b8vrp"
Sep 15 20:57:46.079492 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.079481    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b24f6335-59f4-4b7a-8548-56465ed60966-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b8vrp\" (UID: \"b24f6335-59f4-4b7a-8548-56465ed60966\") " pod="openshift-multus/multus-additional-cni-plugins-b8vrp"
Sep 15 20:57:46.079591 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.079514    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-etc-modprobe-d\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.079591 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.079569    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-sys\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.079662 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.079619    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/928ab144-6fdf-4555-abdf-86b1d0dc8a72-iptables-alerter-script\") pod \"iptables-alerter-h94k8\" (UID: \"928ab144-6fdf-4555-abdf-86b1d0dc8a72\") " pod="openshift-network-operator/iptables-alerter-h94k8"
Sep 15 20:57:46.079662 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.079629    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b24f6335-59f4-4b7a-8548-56465ed60966-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b8vrp\" (UID: \"b24f6335-59f4-4b7a-8548-56465ed60966\") " pod="openshift-multus/multus-additional-cni-plugins-b8vrp"
Sep 15 20:57:46.079732 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.079680    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/88840482-9724-4d2b-8343-3ed2ad33b258-dbus\") pod \"global-pull-secret-syncer-pjz7r\" (UID: \"88840482-9724-4d2b-8343-3ed2ad33b258\") " pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:57:46.079732 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.079725    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-host-slash\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.079795 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.079732    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-multus-socket-dir-parent\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.079924 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.079906    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-node-log\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.079999 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.079920    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-log-socket\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.080049 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.080032    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-system-cni-dir\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.080116 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.080098    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-host-var-lib-cni-multus\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.080548 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.080473    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b24f6335-59f4-4b7a-8548-56465ed60966-cni-binary-copy\") pod \"multus-additional-cni-plugins-b8vrp\" (UID: \"b24f6335-59f4-4b7a-8548-56465ed60966\") " pod="openshift-multus/multus-additional-cni-plugins-b8vrp"
Sep 15 20:57:46.080808 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.080726    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-etc-openvswitch\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.080989 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.078942    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-etc-sysctl-conf\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.081086 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.081061    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fc205950-caf0-4c47-84ce-a8fc54a1afb2-ovnkube-script-lib\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.082404 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.082193    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b24f6335-59f4-4b7a-8548-56465ed60966-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b8vrp\" (UID: \"b24f6335-59f4-4b7a-8548-56465ed60966\") " pod="openshift-multus/multus-additional-cni-plugins-b8vrp"
Sep 15 20:57:46.083525 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.082556    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-var-lib-openvswitch\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.083525 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.082632    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.083525 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.082806    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-multus-cni-dir\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.083525 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.083106    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-multus-daemon-config\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.083525 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.083178    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-host-run-ovn-kubernetes\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.083525 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.083257    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-etc-kubernetes\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.083525 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.083418    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fc205950-caf0-4c47-84ce-a8fc54a1afb2-ovnkube-config\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.083886 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.083633    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-run-ovn\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.083886 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.083642    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-host-cni-netd\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.083886 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.083658    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-etc-kubernetes\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.083886 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.083693    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-run-systemd\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.083886 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.083706    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-host-var-lib-cni-bin\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.083886 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.083757    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-host-var-lib-kubelet\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.083886 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.083815    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-etc-sysconfig\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.084220 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.084053    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b24f6335-59f4-4b7a-8548-56465ed60966-system-cni-dir\") pod \"multus-additional-cni-plugins-b8vrp\" (UID: \"b24f6335-59f4-4b7a-8548-56465ed60966\") " pod="openshift-multus/multus-additional-cni-plugins-b8vrp"
Sep 15 20:57:46.084623 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:46.084600    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:57:46.084710 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:46.084672    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88840482-9724-4d2b-8343-3ed2ad33b258-original-pull-secret podName:88840482-9724-4d2b-8343-3ed2ad33b258 nodeName:}" failed. No retries permitted until 2026-09-15 20:57:46.58465206 +0000 UTC m=+3.126814703 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/88840482-9724-4d2b-8343-3ed2ad33b258-original-pull-secret") pod "global-pull-secret-syncer-pjz7r" (UID: "88840482-9724-4d2b-8343-3ed2ad33b258") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:57:46.084777 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.084727    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-run-openvswitch\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.084831 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.084772    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b24f6335-59f4-4b7a-8548-56465ed60966-cnibin\") pod \"multus-additional-cni-plugins-b8vrp\" (UID: \"b24f6335-59f4-4b7a-8548-56465ed60966\") " pod="openshift-multus/multus-additional-cni-plugins-b8vrp"
Sep 15 20:57:46.085093 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.085062    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-host-cni-bin\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.085163 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.085091    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-host\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.085343 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.085327    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-hostroot\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.085464 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.085444    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/88840482-9724-4d2b-8343-3ed2ad33b258-kubelet-config\") pod \"global-pull-secret-syncer-pjz7r\" (UID: \"88840482-9724-4d2b-8343-3ed2ad33b258\") " pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:57:46.085571 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.085518    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-systemd-units\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.085769 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.085737    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-multus-conf-dir\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.085828 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.085780    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-host-kubelet\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.085880 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.085840    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/88840482-9724-4d2b-8343-3ed2ad33b258-dbus\") pod \"global-pull-secret-syncer-pjz7r\" (UID: \"88840482-9724-4d2b-8343-3ed2ad33b258\") " pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:57:46.085880 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.085858    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b24f6335-59f4-4b7a-8548-56465ed60966-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b8vrp\" (UID: \"b24f6335-59f4-4b7a-8548-56465ed60966\") " pod="openshift-multus/multus-additional-cni-plugins-b8vrp"
Sep 15 20:57:46.085985 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.085879    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-host-run-multus-certs\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.086042 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.085981    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-multus-socket-dir-parent\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.086042 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.086017    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc205950-caf0-4c47-84ce-a8fc54a1afb2-host-run-netns\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.086137 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.086049    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-host-run-k8s-cni-cncf-io\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.086137 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.086128    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-etc-modprobe-d\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.086244 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.086157    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-sys\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.086244 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.086234    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fc205950-caf0-4c47-84ce-a8fc54a1afb2-env-overrides\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.086330 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.086268    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-etc-sysctl-d\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.086774 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.086753    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/928ab144-6fdf-4555-abdf-86b1d0dc8a72-iptables-alerter-script\") pod \"iptables-alerter-h94k8\" (UID: \"928ab144-6fdf-4555-abdf-86b1d0dc8a72\") " pod="openshift-network-operator/iptables-alerter-h94k8"
Sep 15 20:57:46.087978 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.087942    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fc205950-caf0-4c47-84ce-a8fc54a1afb2-ovn-node-metrics-cert\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.090719 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.090665    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb7pb\" (UniqueName: \"kubernetes.io/projected/bbfa79f6-eab8-4b5b-886e-a0a6879b061a-kube-api-access-gb7pb\") pod \"multus-cq8j7\" (UID: \"bbfa79f6-eab8-4b5b-886e-a0a6879b061a\") " pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.090794 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.090737    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-etc-tuned\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.093331 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.092925    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-tmp\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.093331 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:46.093063    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:57:46.093331 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:46.093085    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:57:46.093331 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:46.093104    2564 projected.go:196] Error preparing data for projected volume kube-api-access-dcpx9 for pod openshift-network-diagnostics/network-check-target-m8fpw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:57:46.093331 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:46.093172    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a70ba96c-ccb0-436a-aa3c-e056cb9a76e3-kube-api-access-dcpx9 podName:a70ba96c-ccb0-436a-aa3c-e056cb9a76e3 nodeName:}" failed. No retries permitted until 2026-09-15 20:57:46.593155551 +0000 UTC m=+3.135318188 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dcpx9" (UniqueName: "kubernetes.io/projected/a70ba96c-ccb0-436a-aa3c-e056cb9a76e3-kube-api-access-dcpx9") pod "network-check-target-m8fpw" (UID: "a70ba96c-ccb0-436a-aa3c-e056cb9a76e3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:57:46.094157 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.094139    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqfhs\" (UniqueName: \"kubernetes.io/projected/928ab144-6fdf-4555-abdf-86b1d0dc8a72-kube-api-access-pqfhs\") pod \"iptables-alerter-h94k8\" (UID: \"928ab144-6fdf-4555-abdf-86b1d0dc8a72\") " pod="openshift-network-operator/iptables-alerter-h94k8"
Sep 15 20:57:46.095410 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.095388    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkkcq\" (UniqueName: \"kubernetes.io/projected/b24f6335-59f4-4b7a-8548-56465ed60966-kube-api-access-qkkcq\") pod \"multus-additional-cni-plugins-b8vrp\" (UID: \"b24f6335-59f4-4b7a-8548-56465ed60966\") " pod="openshift-multus/multus-additional-cni-plugins-b8vrp"
Sep 15 20:57:46.095589 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.095388    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvqjl\" (UniqueName: \"kubernetes.io/projected/761b2f93-4f03-4e3e-a1bd-106c32d0d65f-kube-api-access-mvqjl\") pod \"tuned-m94mk\" (UID: \"761b2f93-4f03-4e3e-a1bd-106c32d0d65f\") " pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.097256 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.097234    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt7pz\" (UniqueName: \"kubernetes.io/projected/fc205950-caf0-4c47-84ce-a8fc54a1afb2-kube-api-access-gt7pz\") pod \"ovnkube-node-d4c7k\" (UID: \"fc205950-caf0-4c47-84ce-a8fc54a1afb2\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.165258 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.165223    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gwqhk"
Sep 15 20:57:46.173353 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.173326    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg"
Sep 15 20:57:46.176475 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:46.176443    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc773606_78e0_4ee0_a8ac_29d191e7f936.slice/crio-21f826fa29fdd92ace60ed53ebce42d1536d158b9a203b28f09e793c5e39727b WatchSource:0}: Error finding container 21f826fa29fdd92ace60ed53ebce42d1536d158b9a203b28f09e793c5e39727b: Status 404 returned error can't find the container with id 21f826fa29fdd92ace60ed53ebce42d1536d158b9a203b28f09e793c5e39727b
Sep 15 20:57:46.182808 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.182637    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mt7b6"
Sep 15 20:57:46.191079 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.191052    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cdrtr"
Sep 15 20:57:46.193850 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:46.193821    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c4d34d1_b3a5_4974_9db9_0ccf8786db47.slice/crio-35ed5f55ba4a6ec88336c2b9c3a11ebd25fa6a977175bd5fbb3f7987753f74d0 WatchSource:0}: Error finding container 35ed5f55ba4a6ec88336c2b9c3a11ebd25fa6a977175bd5fbb3f7987753f74d0: Status 404 returned error can't find the container with id 35ed5f55ba4a6ec88336c2b9c3a11ebd25fa6a977175bd5fbb3f7987753f74d0
Sep 15 20:57:46.201151 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.201127    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h94k8"
Sep 15 20:57:46.202720 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:46.202693    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafadf0fd_9154_4a32_8abe_d8b29dfcaa90.slice/crio-242bd7e89edf896b645264a01acad42232a7e3e12e403121beec09909d2745bc WatchSource:0}: Error finding container 242bd7e89edf896b645264a01acad42232a7e3e12e403121beec09909d2745bc: Status 404 returned error can't find the container with id 242bd7e89edf896b645264a01acad42232a7e3e12e403121beec09909d2745bc
Sep 15 20:57:46.208701 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.208676    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m94mk"
Sep 15 20:57:46.217601 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.217574    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b8vrp"
Sep 15 20:57:46.222333 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:46.222294    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod761b2f93_4f03_4e3e_a1bd_106c32d0d65f.slice/crio-3b179f8f55ee1464c57776a7b1c508cf82d0fa382484d87f85228a74029810f2 WatchSource:0}: Error finding container 3b179f8f55ee1464c57776a7b1c508cf82d0fa382484d87f85228a74029810f2: Status 404 returned error can't find the container with id 3b179f8f55ee1464c57776a7b1c508cf82d0fa382484d87f85228a74029810f2
Sep 15 20:57:46.226661 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.226625    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cq8j7"
Sep 15 20:57:46.233676 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.233508    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:57:46.235090 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:46.235062    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb24f6335_59f4_4b7a_8548_56465ed60966.slice/crio-107b1d8f4fff1232c8276e0628724cf17831ae6d6462a0c26b20c9f431aea91a WatchSource:0}: Error finding container 107b1d8f4fff1232c8276e0628724cf17831ae6d6462a0c26b20c9f431aea91a: Status 404 returned error can't find the container with id 107b1d8f4fff1232c8276e0628724cf17831ae6d6462a0c26b20c9f431aea91a
Sep 15 20:57:46.238320 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:46.238302    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbfa79f6_eab8_4b5b_886e_a0a6879b061a.slice/crio-f92e948dc8cc07fd766c0af19ae2515f3fdb094b18e36e2bcae28d9a4e68729b WatchSource:0}: Error finding container f92e948dc8cc07fd766c0af19ae2515f3fdb094b18e36e2bcae28d9a4e68729b: Status 404 returned error can't find the container with id f92e948dc8cc07fd766c0af19ae2515f3fdb094b18e36e2bcae28d9a4e68729b
Sep 15 20:57:46.245129 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:57:46.245093    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc205950_caf0_4c47_84ce_a8fc54a1afb2.slice/crio-c3387bd7984138f7d2f828584de01087638bab616ccdb90992d4157b23ebd7e3 WatchSource:0}: Error finding container c3387bd7984138f7d2f828584de01087638bab616ccdb90992d4157b23ebd7e3: Status 404 returned error can't find the container with id c3387bd7984138f7d2f828584de01087638bab616ccdb90992d4157b23ebd7e3
Sep 15 20:57:46.483010 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.482909    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs\") pod \"network-metrics-daemon-4dws2\" (UID: \"ce801675-8966-401c-b202-6631f49bdea0\") " pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:57:46.483177 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:46.483044    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:57:46.483177 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:46.483117    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs podName:ce801675-8966-401c-b202-6631f49bdea0 nodeName:}" failed. No retries permitted until 2026-09-15 20:57:47.483096762 +0000 UTC m=+4.025259391 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs") pod "network-metrics-daemon-4dws2" (UID: "ce801675-8966-401c-b202-6631f49bdea0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:57:46.684671 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.684630    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcpx9\" (UniqueName: \"kubernetes.io/projected/a70ba96c-ccb0-436a-aa3c-e056cb9a76e3-kube-api-access-dcpx9\") pod \"network-check-target-m8fpw\" (UID: \"a70ba96c-ccb0-436a-aa3c-e056cb9a76e3\") " pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:57:46.684671 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.684678    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88840482-9724-4d2b-8343-3ed2ad33b258-original-pull-secret\") pod \"global-pull-secret-syncer-pjz7r\" (UID: \"88840482-9724-4d2b-8343-3ed2ad33b258\") " pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:57:46.684940 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:46.684804    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:57:46.684940 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:46.684806    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:57:46.684940 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:46.684829    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:57:46.684940 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:46.684841    2564 projected.go:196] Error preparing data for projected volume kube-api-access-dcpx9 for pod openshift-network-diagnostics/network-check-target-m8fpw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:57:46.684940 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:46.684874    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88840482-9724-4d2b-8343-3ed2ad33b258-original-pull-secret podName:88840482-9724-4d2b-8343-3ed2ad33b258 nodeName:}" failed. No retries permitted until 2026-09-15 20:57:47.684857676 +0000 UTC m=+4.227020309 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/88840482-9724-4d2b-8343-3ed2ad33b258-original-pull-secret") pod "global-pull-secret-syncer-pjz7r" (UID: "88840482-9724-4d2b-8343-3ed2ad33b258") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:57:46.684940 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:46.684890    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a70ba96c-ccb0-436a-aa3c-e056cb9a76e3-kube-api-access-dcpx9 podName:a70ba96c-ccb0-436a-aa3c-e056cb9a76e3 nodeName:}" failed. No retries permitted until 2026-09-15 20:57:47.684883926 +0000 UTC m=+4.227046539 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dcpx9" (UniqueName: "kubernetes.io/projected/a70ba96c-ccb0-436a-aa3c-e056cb9a76e3-kube-api-access-dcpx9") pod "network-check-target-m8fpw" (UID: "a70ba96c-ccb0-436a-aa3c-e056cb9a76e3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:57:46.902254 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.902197    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 20:52:44 +0000 UTC" deadline="2028-06-16 17:58:27.364022384 +0000 UTC"
Sep 15 20:57:46.902254 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.902252    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15357h0m40.461775102s"
Sep 15 20:57:46.983256 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.983219    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8vrp" event={"ID":"b24f6335-59f4-4b7a-8548-56465ed60966","Type":"ContainerStarted","Data":"107b1d8f4fff1232c8276e0628724cf17831ae6d6462a0c26b20c9f431aea91a"}
Sep 15 20:57:46.984535 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.984470    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m94mk" event={"ID":"761b2f93-4f03-4e3e-a1bd-106c32d0d65f","Type":"ContainerStarted","Data":"3b179f8f55ee1464c57776a7b1c508cf82d0fa382484d87f85228a74029810f2"}
Sep 15 20:57:46.985820 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.985779    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cdrtr" event={"ID":"afadf0fd-9154-4a32-8abe-d8b29dfcaa90","Type":"ContainerStarted","Data":"242bd7e89edf896b645264a01acad42232a7e3e12e403121beec09909d2745bc"}
Sep 15 20:57:46.987106 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.987079    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gwqhk" event={"ID":"bc773606-78e0-4ee0-a8ac-29d191e7f936","Type":"ContainerStarted","Data":"21f826fa29fdd92ace60ed53ebce42d1536d158b9a203b28f09e793c5e39727b"}
Sep 15 20:57:46.988447 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.988423    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k" event={"ID":"fc205950-caf0-4c47-84ce-a8fc54a1afb2","Type":"ContainerStarted","Data":"c3387bd7984138f7d2f828584de01087638bab616ccdb90992d4157b23ebd7e3"}
Sep 15 20:57:46.989737 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.989713    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cq8j7" event={"ID":"bbfa79f6-eab8-4b5b-886e-a0a6879b061a","Type":"ContainerStarted","Data":"f92e948dc8cc07fd766c0af19ae2515f3fdb094b18e36e2bcae28d9a4e68729b"}
Sep 15 20:57:46.990999 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.990974    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h94k8" event={"ID":"928ab144-6fdf-4555-abdf-86b1d0dc8a72","Type":"ContainerStarted","Data":"d64232a72ec5638c1d6d23a6b21781eb67608af9c8b16e5a477831ca9eb724a8"}
Sep 15 20:57:46.992361 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.992335    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mt7b6" event={"ID":"7c4d34d1-b3a5-4974-9db9-0ccf8786db47","Type":"ContainerStarted","Data":"35ed5f55ba4a6ec88336c2b9c3a11ebd25fa6a977175bd5fbb3f7987753f74d0"}
Sep 15 20:57:46.993583 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:46.993564    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg" event={"ID":"0bf2384a-bc40-40a7-a139-98a53ebf6b68","Type":"ContainerStarted","Data":"915d7233f724838e4e83109d9eaa4502c96b9d7072449f7c6117d07e79f42b1f"}
Sep 15 20:57:47.490988 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:47.490948    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs\") pod \"network-metrics-daemon-4dws2\" (UID: \"ce801675-8966-401c-b202-6631f49bdea0\") " pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:57:47.491178 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:47.491155    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:57:47.491263 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:47.491240    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs podName:ce801675-8966-401c-b202-6631f49bdea0 nodeName:}" failed. No retries permitted until 2026-09-15 20:57:49.491218939 +0000 UTC m=+6.033381573 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs") pod "network-metrics-daemon-4dws2" (UID: "ce801675-8966-401c-b202-6631f49bdea0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:57:47.693868 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:47.693826    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcpx9\" (UniqueName: \"kubernetes.io/projected/a70ba96c-ccb0-436a-aa3c-e056cb9a76e3-kube-api-access-dcpx9\") pod \"network-check-target-m8fpw\" (UID: \"a70ba96c-ccb0-436a-aa3c-e056cb9a76e3\") " pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:57:47.693970 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:47.693885    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88840482-9724-4d2b-8343-3ed2ad33b258-original-pull-secret\") pod \"global-pull-secret-syncer-pjz7r\" (UID: \"88840482-9724-4d2b-8343-3ed2ad33b258\") " pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:57:47.694033 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:47.694013    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:57:47.694090 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:47.694080    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88840482-9724-4d2b-8343-3ed2ad33b258-original-pull-secret podName:88840482-9724-4d2b-8343-3ed2ad33b258 nodeName:}" failed. No retries permitted until 2026-09-15 20:57:49.694062599 +0000 UTC m=+6.236225215 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/88840482-9724-4d2b-8343-3ed2ad33b258-original-pull-secret") pod "global-pull-secret-syncer-pjz7r" (UID: "88840482-9724-4d2b-8343-3ed2ad33b258") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:57:47.694543 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:47.694481    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:57:47.694543 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:47.694529    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:57:47.694543 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:47.694543    2564 projected.go:196] Error preparing data for projected volume kube-api-access-dcpx9 for pod openshift-network-diagnostics/network-check-target-m8fpw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:57:47.694711 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:47.694594    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a70ba96c-ccb0-436a-aa3c-e056cb9a76e3-kube-api-access-dcpx9 podName:a70ba96c-ccb0-436a-aa3c-e056cb9a76e3 nodeName:}" failed. No retries permitted until 2026-09-15 20:57:49.694577548 +0000 UTC m=+6.236740161 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dcpx9" (UniqueName: "kubernetes.io/projected/a70ba96c-ccb0-436a-aa3c-e056cb9a76e3-kube-api-access-dcpx9") pod "network-check-target-m8fpw" (UID: "a70ba96c-ccb0-436a-aa3c-e056cb9a76e3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:57:47.968928 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:47.968836    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:57:47.969656 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:47.968989    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjz7r" podUID="88840482-9724-4d2b-8343-3ed2ad33b258"
Sep 15 20:57:47.969656 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:47.969407    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:57:47.969656 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:47.969527    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dws2" podUID="ce801675-8966-401c-b202-6631f49bdea0"
Sep 15 20:57:47.969656 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:47.969595    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:57:47.969881 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:47.969667    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m8fpw" podUID="a70ba96c-ccb0-436a-aa3c-e056cb9a76e3"
Sep 15 20:57:48.012881 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:48.012841    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-221.ec2.internal" event={"ID":"6ea81e5eea0bb90932530ee9266ef4ab","Type":"ContainerStarted","Data":"88710dfaa50cca51ff387e030801cf395359fd779cd09ab6674a6d8dd01b2e9b"}
Sep 15 20:57:49.031369 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:49.030157    2564 generic.go:356] "Generic (PLEG): container finished" podID="7e947b5ccc9cdb54b46a5b4771d1b37b" containerID="fa47740f94b8d87ba0c666a108d5e41ebd8c6cb78855956d8bf66db77c0bf913" exitCode=0
Sep 15 20:57:49.031369 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:49.031271    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-221.ec2.internal" event={"ID":"7e947b5ccc9cdb54b46a5b4771d1b37b","Type":"ContainerDied","Data":"fa47740f94b8d87ba0c666a108d5e41ebd8c6cb78855956d8bf66db77c0bf913"}
Sep 15 20:57:49.046953 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:49.046879    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-221.ec2.internal" podStartSLOduration=5.046853276 podStartE2EDuration="5.046853276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:57:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:57:48.02828073 +0000 UTC m=+4.570443361" watchObservedRunningTime="2026-09-15 20:57:49.046853276 +0000 UTC m=+5.589015910"
Sep 15 20:57:49.511314 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:49.511223    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs\") pod \"network-metrics-daemon-4dws2\" (UID: \"ce801675-8966-401c-b202-6631f49bdea0\") " pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:57:49.511518 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:49.511418    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:57:49.511518 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:49.511483    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs podName:ce801675-8966-401c-b202-6631f49bdea0 nodeName:}" failed. No retries permitted until 2026-09-15 20:57:53.511464364 +0000 UTC m=+10.053626989 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs") pod "network-metrics-daemon-4dws2" (UID: "ce801675-8966-401c-b202-6631f49bdea0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:57:49.713020 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:49.712978    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcpx9\" (UniqueName: \"kubernetes.io/projected/a70ba96c-ccb0-436a-aa3c-e056cb9a76e3-kube-api-access-dcpx9\") pod \"network-check-target-m8fpw\" (UID: \"a70ba96c-ccb0-436a-aa3c-e056cb9a76e3\") " pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:57:49.713205 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:49.713033    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88840482-9724-4d2b-8343-3ed2ad33b258-original-pull-secret\") pod \"global-pull-secret-syncer-pjz7r\" (UID: \"88840482-9724-4d2b-8343-3ed2ad33b258\") " pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:57:49.713205 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:49.713158    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:57:49.713304 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:49.713222    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88840482-9724-4d2b-8343-3ed2ad33b258-original-pull-secret podName:88840482-9724-4d2b-8343-3ed2ad33b258 nodeName:}" failed. No retries permitted until 2026-09-15 20:57:53.713204032 +0000 UTC m=+10.255366645 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/88840482-9724-4d2b-8343-3ed2ad33b258-original-pull-secret") pod "global-pull-secret-syncer-pjz7r" (UID: "88840482-9724-4d2b-8343-3ed2ad33b258") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:57:49.713687 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:49.713662    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:57:49.713792 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:49.713692    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:57:49.713792 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:49.713705    2564 projected.go:196] Error preparing data for projected volume kube-api-access-dcpx9 for pod openshift-network-diagnostics/network-check-target-m8fpw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:57:49.713792 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:49.713752    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a70ba96c-ccb0-436a-aa3c-e056cb9a76e3-kube-api-access-dcpx9 podName:a70ba96c-ccb0-436a-aa3c-e056cb9a76e3 nodeName:}" failed. No retries permitted until 2026-09-15 20:57:53.713736532 +0000 UTC m=+10.255899164 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dcpx9" (UniqueName: "kubernetes.io/projected/a70ba96c-ccb0-436a-aa3c-e056cb9a76e3-kube-api-access-dcpx9") pod "network-check-target-m8fpw" (UID: "a70ba96c-ccb0-436a-aa3c-e056cb9a76e3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:57:49.967763 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:49.967614    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:57:49.967763 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:49.967647    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:57:49.967763 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:49.967743    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjz7r" podUID="88840482-9724-4d2b-8343-3ed2ad33b258"
Sep 15 20:57:49.968204 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:49.968146    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m8fpw" podUID="a70ba96c-ccb0-436a-aa3c-e056cb9a76e3"
Sep 15 20:57:49.968292 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:49.968215    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:57:49.968348 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:49.968304    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dws2" podUID="ce801675-8966-401c-b202-6631f49bdea0"
Sep 15 20:57:51.968976 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:51.968946    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:57:51.969418 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:51.969087    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjz7r" podUID="88840482-9724-4d2b-8343-3ed2ad33b258"
Sep 15 20:57:51.969418 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:51.969148    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:57:51.969418 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:51.969259    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dws2" podUID="ce801675-8966-401c-b202-6631f49bdea0"
Sep 15 20:57:51.969418 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:51.969301    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:57:51.969418 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:51.969355    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m8fpw" podUID="a70ba96c-ccb0-436a-aa3c-e056cb9a76e3"
Sep 15 20:57:53.547955 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:53.547828    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs\") pod \"network-metrics-daemon-4dws2\" (UID: \"ce801675-8966-401c-b202-6631f49bdea0\") " pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:57:53.548396 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:53.548001    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:57:53.548396 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:53.548071    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs podName:ce801675-8966-401c-b202-6631f49bdea0 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:01.548049666 +0000 UTC m=+18.090212292 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs") pod "network-metrics-daemon-4dws2" (UID: "ce801675-8966-401c-b202-6631f49bdea0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:57:53.749000 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:53.748961    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcpx9\" (UniqueName: \"kubernetes.io/projected/a70ba96c-ccb0-436a-aa3c-e056cb9a76e3-kube-api-access-dcpx9\") pod \"network-check-target-m8fpw\" (UID: \"a70ba96c-ccb0-436a-aa3c-e056cb9a76e3\") " pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:57:53.749167 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:53.749016    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88840482-9724-4d2b-8343-3ed2ad33b258-original-pull-secret\") pod \"global-pull-secret-syncer-pjz7r\" (UID: \"88840482-9724-4d2b-8343-3ed2ad33b258\") " pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:57:53.749167 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:53.749144    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:57:53.749167 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:53.749167    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:57:53.749335 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:53.749169    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:57:53.749335 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:53.749177    2564 projected.go:196] Error preparing data for projected volume kube-api-access-dcpx9 for pod openshift-network-diagnostics/network-check-target-m8fpw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:57:53.749335 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:53.749242    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88840482-9724-4d2b-8343-3ed2ad33b258-original-pull-secret podName:88840482-9724-4d2b-8343-3ed2ad33b258 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:01.749221163 +0000 UTC m=+18.291383788 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/88840482-9724-4d2b-8343-3ed2ad33b258-original-pull-secret") pod "global-pull-secret-syncer-pjz7r" (UID: "88840482-9724-4d2b-8343-3ed2ad33b258") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:57:53.749335 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:53.749263    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a70ba96c-ccb0-436a-aa3c-e056cb9a76e3-kube-api-access-dcpx9 podName:a70ba96c-ccb0-436a-aa3c-e056cb9a76e3 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:01.749252838 +0000 UTC m=+18.291415457 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dcpx9" (UniqueName: "kubernetes.io/projected/a70ba96c-ccb0-436a-aa3c-e056cb9a76e3-kube-api-access-dcpx9") pod "network-check-target-m8fpw" (UID: "a70ba96c-ccb0-436a-aa3c-e056cb9a76e3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:57:53.967387 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:53.967303    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:57:53.967583 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:53.967418    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:57:53.967583 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:53.967441    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:57:53.967583 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:53.967414    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjz7r" podUID="88840482-9724-4d2b-8343-3ed2ad33b258"
Sep 15 20:57:53.967583 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:53.967550    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dws2" podUID="ce801675-8966-401c-b202-6631f49bdea0"
Sep 15 20:57:53.967774 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:53.967628    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m8fpw" podUID="a70ba96c-ccb0-436a-aa3c-e056cb9a76e3"
Sep 15 20:57:55.967537 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:55.967412    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:57:55.968559 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:55.967417    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:57:55.971035 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:55.969744    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m8fpw" podUID="a70ba96c-ccb0-436a-aa3c-e056cb9a76e3"
Sep 15 20:57:55.971035 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:55.970216    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:57:55.971035 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:55.970389    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjz7r" podUID="88840482-9724-4d2b-8343-3ed2ad33b258"
Sep 15 20:57:55.971035 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:55.970527    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dws2" podUID="ce801675-8966-401c-b202-6631f49bdea0"
Sep 15 20:57:57.966902 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:57.966863    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:57:57.967343 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:57.966871    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:57:57.967343 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:57.966988    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjz7r" podUID="88840482-9724-4d2b-8343-3ed2ad33b258"
Sep 15 20:57:57.967343 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:57.967016    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:57:57.967343 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:57.967116    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dws2" podUID="ce801675-8966-401c-b202-6631f49bdea0"
Sep 15 20:57:57.967343 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:57.967190    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m8fpw" podUID="a70ba96c-ccb0-436a-aa3c-e056cb9a76e3"
Sep 15 20:57:59.966773 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:59.966727    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:57:59.967231 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:59.966778    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:57:59.967231 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:57:59.966838    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:57:59.967231 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:59.966863    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjz7r" podUID="88840482-9724-4d2b-8343-3ed2ad33b258"
Sep 15 20:57:59.967231 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:59.966931    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m8fpw" podUID="a70ba96c-ccb0-436a-aa3c-e056cb9a76e3"
Sep 15 20:57:59.967231 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:57:59.967029    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dws2" podUID="ce801675-8966-401c-b202-6631f49bdea0"
Sep 15 20:58:01.616916 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:01.616876    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs\") pod \"network-metrics-daemon-4dws2\" (UID: \"ce801675-8966-401c-b202-6631f49bdea0\") " pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:58:01.617454 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:01.617050    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:58:01.617454 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:01.617130    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs podName:ce801675-8966-401c-b202-6631f49bdea0 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:17.617112526 +0000 UTC m=+34.159275144 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs") pod "network-metrics-daemon-4dws2" (UID: "ce801675-8966-401c-b202-6631f49bdea0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 20:58:01.818370 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:01.818326    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcpx9\" (UniqueName: \"kubernetes.io/projected/a70ba96c-ccb0-436a-aa3c-e056cb9a76e3-kube-api-access-dcpx9\") pod \"network-check-target-m8fpw\" (UID: \"a70ba96c-ccb0-436a-aa3c-e056cb9a76e3\") " pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:58:01.818581 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:01.818383    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88840482-9724-4d2b-8343-3ed2ad33b258-original-pull-secret\") pod \"global-pull-secret-syncer-pjz7r\" (UID: \"88840482-9724-4d2b-8343-3ed2ad33b258\") " pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:58:01.818581 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:01.818479    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 20:58:01.818581 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:01.818511    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 20:58:01.818581 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:01.818524    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 20:58:01.818581 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:01.818539    2564 projected.go:196] Error preparing data for projected volume kube-api-access-dcpx9 for pod openshift-network-diagnostics/network-check-target-m8fpw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:58:01.818581 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:01.818572    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88840482-9724-4d2b-8343-3ed2ad33b258-original-pull-secret podName:88840482-9724-4d2b-8343-3ed2ad33b258 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:17.818555701 +0000 UTC m=+34.360718334 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/88840482-9724-4d2b-8343-3ed2ad33b258-original-pull-secret") pod "global-pull-secret-syncer-pjz7r" (UID: "88840482-9724-4d2b-8343-3ed2ad33b258") : object "kube-system"/"original-pull-secret" not registered
Sep 15 20:58:01.818906 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:01.818588    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a70ba96c-ccb0-436a-aa3c-e056cb9a76e3-kube-api-access-dcpx9 podName:a70ba96c-ccb0-436a-aa3c-e056cb9a76e3 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:17.818577939 +0000 UTC m=+34.360740564 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dcpx9" (UniqueName: "kubernetes.io/projected/a70ba96c-ccb0-436a-aa3c-e056cb9a76e3-kube-api-access-dcpx9") pod "network-check-target-m8fpw" (UID: "a70ba96c-ccb0-436a-aa3c-e056cb9a76e3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 20:58:01.966260 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:01.966158    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:58:01.966429 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:01.966158    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:58:01.966429 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:01.966317    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dws2" podUID="ce801675-8966-401c-b202-6631f49bdea0"
Sep 15 20:58:01.966557 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:01.966441    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m8fpw" podUID="a70ba96c-ccb0-436a-aa3c-e056cb9a76e3"
Sep 15 20:58:01.966557 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:01.966480    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:58:01.966654 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:01.966589    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjz7r" podUID="88840482-9724-4d2b-8343-3ed2ad33b258"
Sep 15 20:58:03.967768 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:03.967727    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:58:03.968244 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:03.967823    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:58:03.968244 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:03.967857    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:58:03.968244 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:03.967867    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjz7r" podUID="88840482-9724-4d2b-8343-3ed2ad33b258"
Sep 15 20:58:03.968244 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:03.967936    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m8fpw" podUID="a70ba96c-ccb0-436a-aa3c-e056cb9a76e3"
Sep 15 20:58:03.968244 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:03.968082    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dws2" podUID="ce801675-8966-401c-b202-6631f49bdea0"
Sep 15 20:58:05.070109 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:05.070025    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-221.ec2.internal" event={"ID":"7e947b5ccc9cdb54b46a5b4771d1b37b","Type":"ContainerStarted","Data":"cab6dc62d2773e91bfa1ae07c158038985711a31007afc6072a8a5470b411281"}
Sep 15 20:58:05.079466 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:05.079395    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cdrtr" event={"ID":"afadf0fd-9154-4a32-8abe-d8b29dfcaa90","Type":"ContainerStarted","Data":"b8f3d473d51ecea2cf627499fab1b16132a030d174e4a2d196b0f7ae490be92f"}
Sep 15 20:58:05.086582 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:05.086508    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gwqhk" event={"ID":"bc773606-78e0-4ee0-a8ac-29d191e7f936","Type":"ContainerStarted","Data":"b48ebebc0fc380593521c80acdfd6f530d90c380a4faa34cfaaf777e9f7afeb5"}
Sep 15 20:58:05.101183 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:05.101090    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gwqhk" podStartSLOduration=6.624981756 podStartE2EDuration="21.101067188s" totalImagesPullingTime="14.476085432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:57:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:05.099913936 +0000 UTC m=+21.642076574" watchObservedRunningTime="2026-09-15 20:58:05.101067188 +0000 UTC m=+21.643229825"
Sep 15 20:58:05.966523 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:05.966188    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:58:05.966745 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:05.966191    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:58:05.966745 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:05.966613    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dws2" podUID="ce801675-8966-401c-b202-6631f49bdea0"
Sep 15 20:58:05.966745 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:05.966196    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:58:05.966745 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:05.966683    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjz7r" podUID="88840482-9724-4d2b-8343-3ed2ad33b258"
Sep 15 20:58:05.966945 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:05.966769    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m8fpw" podUID="a70ba96c-ccb0-436a-aa3c-e056cb9a76e3"
Sep 15 20:58:06.090761 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.090727    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg" event={"ID":"0bf2384a-bc40-40a7-a139-98a53ebf6b68","Type":"ContainerStarted","Data":"d7ac359c56b9f61798b425fd2f375c17870eb714e928b53970c6e9c4cf6ee5a2"}
Sep 15 20:58:06.092390 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.092362    2564 generic.go:356] "Generic (PLEG): container finished" podID="b24f6335-59f4-4b7a-8548-56465ed60966" containerID="b1cdd2f5f3d080000343f6b586d0f6adf662398b4f2ea844442b53d479e4f2f4" exitCode=0
Sep 15 20:58:06.092526 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.092442    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8vrp" event={"ID":"b24f6335-59f4-4b7a-8548-56465ed60966","Type":"ContainerDied","Data":"b1cdd2f5f3d080000343f6b586d0f6adf662398b4f2ea844442b53d479e4f2f4"}
Sep 15 20:58:06.094183 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.094159    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m94mk" event={"ID":"761b2f93-4f03-4e3e-a1bd-106c32d0d65f","Type":"ContainerStarted","Data":"1fc00457cfcb401917c998bbb64406fe27e136ab34738fe6ad0de099a87267fe"}
Sep 15 20:58:06.097283 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.097266    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4c7k_fc205950-caf0-4c47-84ce-a8fc54a1afb2/ovn-acl-logging/0.log"
Sep 15 20:58:06.097772 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.097753    2564 generic.go:356] "Generic (PLEG): container finished" podID="fc205950-caf0-4c47-84ce-a8fc54a1afb2" containerID="47f607c932f9c83e32dbb2daf8205edf29d46d914b83dfd87714888296d8e55b" exitCode=1
Sep 15 20:58:06.097875 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.097788    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k" event={"ID":"fc205950-caf0-4c47-84ce-a8fc54a1afb2","Type":"ContainerStarted","Data":"d030c553e7c80814f7fea4e83066e3672c3f8e4b2546feae9be07d845a3eceb0"}
Sep 15 20:58:06.097875 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.097855    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k" event={"ID":"fc205950-caf0-4c47-84ce-a8fc54a1afb2","Type":"ContainerStarted","Data":"09bdb92ed069adfc0d9cfe5c8ea4af4657d2ae7a0fc80dd2a8ccdf996df6947e"}
Sep 15 20:58:06.097969 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.097905    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k" event={"ID":"fc205950-caf0-4c47-84ce-a8fc54a1afb2","Type":"ContainerStarted","Data":"ed4ed4b58d7dd30b306cc8ae0eb4348eacab6c9c053f07d16f722d679d4ccb06"}
Sep 15 20:58:06.097969 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.097952    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k" event={"ID":"fc205950-caf0-4c47-84ce-a8fc54a1afb2","Type":"ContainerStarted","Data":"a260f074ee2a9a7bf637616c6057e4d906f56e67b830d77dc366e44b4dba3bfc"}
Sep 15 20:58:06.098032 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.097989    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k" event={"ID":"fc205950-caf0-4c47-84ce-a8fc54a1afb2","Type":"ContainerDied","Data":"47f607c932f9c83e32dbb2daf8205edf29d46d914b83dfd87714888296d8e55b"}
Sep 15 20:58:06.098032 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.098027    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k" event={"ID":"fc205950-caf0-4c47-84ce-a8fc54a1afb2","Type":"ContainerStarted","Data":"8869b88fefa8f434a53c9cfc111cb0aaa8039ab2238bc957b8677b9ed2186a80"}
Sep 15 20:58:06.099430 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.099406    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cq8j7" event={"ID":"bbfa79f6-eab8-4b5b-886e-a0a6879b061a","Type":"ContainerStarted","Data":"b4821656f589315ca23228bf6b490e04ad920d4007bc3a9c35d2115bd53bf128"}
Sep 15 20:58:06.101160 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.101139    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mt7b6" event={"ID":"7c4d34d1-b3a5-4974-9db9-0ccf8786db47","Type":"ContainerStarted","Data":"57d7c245b9977037337e476214dc6dde18d31a6e78a946ee13717686b4694cda"}
Sep 15 20:58:06.128266 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.128208    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cq8j7" podStartSLOduration=3.074185427 podStartE2EDuration="22.12818868s" totalImagesPullingTime="19.054003253s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:57:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:06.127289508 +0000 UTC m=+22.669452144" watchObservedRunningTime="2026-09-15 20:58:06.12818868 +0000 UTC m=+22.670351315"
Sep 15 20:58:06.152190 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.152138    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mt7b6" podStartSLOduration=3.5028152439999998 podStartE2EDuration="22.152120378s" totalImagesPullingTime="18.649305134s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:57:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:06.152023324 +0000 UTC m=+22.694185961" watchObservedRunningTime="2026-09-15 20:58:06.152120378 +0000 UTC m=+22.694283012"
Sep 15 20:58:06.184451 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.184386    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cdrtr" podStartSLOduration=7.732154319 podStartE2EDuration="22.184367182s" totalImagesPullingTime="14.452212863s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:57:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:06.184266889 +0000 UTC m=+22.726429547" watchObservedRunningTime="2026-09-15 20:58:06.184367182 +0000 UTC m=+22.726529820"
Sep 15 20:58:06.202724 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.202683    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-m94mk" podStartSLOduration=3.505235431 podStartE2EDuration="22.202666614s" totalImagesPullingTime="18.697431183s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:57:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:06.202239824 +0000 UTC m=+22.744402460" watchObservedRunningTime="2026-09-15 20:58:06.202666614 +0000 UTC m=+22.744829286"
Sep 15 20:58:06.313620 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.313529    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gwqhk"
Sep 15 20:58:06.314323 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.314265    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gwqhk"
Sep 15 20:58:06.332411 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.332359    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-221.ec2.internal" podStartSLOduration=22.332340637 podStartE2EDuration="22.332340637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:57:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:06.220231946 +0000 UTC m=+22.762394592" watchObservedRunningTime="2026-09-15 20:58:06.332340637 +0000 UTC m=+22.874503271"
Sep 15 20:58:06.367349 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.367328    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 15 20:58:06.944863 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.944684    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-15T20:58:06.367344639Z","UUID":"66d98fef-2ce1-4f9e-a9ac-956308ec2434","Handler":null,"Name":"","Endpoint":""}
Sep 15 20:58:06.946160 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.946141    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 15 20:58:06.946253 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:06.946172    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 15 20:58:07.105839 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:07.105798    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h94k8" event={"ID":"928ab144-6fdf-4555-abdf-86b1d0dc8a72","Type":"ContainerStarted","Data":"9fc3544da3841b134fb5ac8aa4d485feb5c7fbcfac93756367ba69a44aa84b43"}
Sep 15 20:58:07.108269 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:07.108237    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg" event={"ID":"0bf2384a-bc40-40a7-a139-98a53ebf6b68","Type":"ContainerStarted","Data":"91592fbc645dc9667eebf6ab444389f8253f793134d6826422748eedf92f21e6"}
Sep 15 20:58:07.966964 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:07.966922    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:58:07.966964 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:07.966938    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:58:07.967226 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:07.967062    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:58:07.967226 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:07.967059    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m8fpw" podUID="a70ba96c-ccb0-436a-aa3c-e056cb9a76e3"
Sep 15 20:58:07.967226 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:07.967184    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dws2" podUID="ce801675-8966-401c-b202-6631f49bdea0"
Sep 15 20:58:07.967353 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:07.967278    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjz7r" podUID="88840482-9724-4d2b-8343-3ed2ad33b258"
Sep 15 20:58:08.113464 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:08.113436    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4c7k_fc205950-caf0-4c47-84ce-a8fc54a1afb2/ovn-acl-logging/0.log"
Sep 15 20:58:08.113897 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:08.113875    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k" event={"ID":"fc205950-caf0-4c47-84ce-a8fc54a1afb2","Type":"ContainerStarted","Data":"66c62364f842c4d7576a8d0998a72024bee333ad2a72b2b118d998144ab1518d"}
Sep 15 20:58:08.115948 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:08.115924    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg" event={"ID":"0bf2384a-bc40-40a7-a139-98a53ebf6b68","Type":"ContainerStarted","Data":"26deb5157b2ea9a29a9b50cfc42469270911318ee57a0626bef5d9eb54ee07c8"}
Sep 15 20:58:08.116215 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:08.116200    2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 15 20:58:08.132775 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:08.132729    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-h94k8" podStartSLOduration=5.44084994 podStartE2EDuration="24.132714206s" totalImagesPullingTime="18.691864266s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:57:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:07.124784642 +0000 UTC m=+23.666947278" watchObservedRunningTime="2026-09-15 20:58:08.132714206 +0000 UTC m=+24.674876852"
Sep 15 20:58:08.133208 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:08.133182    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x87mg" podStartSLOduration=3.196895006 podStartE2EDuration="24.133171844s" totalImagesPullingTime="20.936276838s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:57:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:08.132226174 +0000 UTC m=+24.674388821" watchObservedRunningTime="2026-09-15 20:58:08.133171844 +0000 UTC m=+24.675334480"
Sep 15 20:58:09.966938 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:09.966863    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:58:09.967360 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:09.966873    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:58:09.967360 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:09.966985    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjz7r" podUID="88840482-9724-4d2b-8343-3ed2ad33b258"
Sep 15 20:58:09.967360 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:09.966999    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:58:09.967360 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:09.967094    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dws2" podUID="ce801675-8966-401c-b202-6631f49bdea0"
Sep 15 20:58:09.967360 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:09.967171    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m8fpw" podUID="a70ba96c-ccb0-436a-aa3c-e056cb9a76e3"
Sep 15 20:58:11.966860 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:11.966620    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:58:11.967676 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:11.966620    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:58:11.967676 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:11.966620    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:58:11.967676 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:11.967012    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dws2" podUID="ce801675-8966-401c-b202-6631f49bdea0"
Sep 15 20:58:11.967676 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:11.966888    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjz7r" podUID="88840482-9724-4d2b-8343-3ed2ad33b258"
Sep 15 20:58:11.967676 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:11.967106    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m8fpw" podUID="a70ba96c-ccb0-436a-aa3c-e056cb9a76e3"
Sep 15 20:58:12.128044 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:12.128011    2564 generic.go:356] "Generic (PLEG): container finished" podID="b24f6335-59f4-4b7a-8548-56465ed60966" containerID="79a51e42661e84dfd8b51ee6dba207962e678e7397fc85c6d4687baf74a9cb5f" exitCode=0
Sep 15 20:58:12.128199 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:12.128095    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8vrp" event={"ID":"b24f6335-59f4-4b7a-8548-56465ed60966","Type":"ContainerDied","Data":"79a51e42661e84dfd8b51ee6dba207962e678e7397fc85c6d4687baf74a9cb5f"}
Sep 15 20:58:12.131881 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:12.131864    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4c7k_fc205950-caf0-4c47-84ce-a8fc54a1afb2/ovn-acl-logging/0.log"
Sep 15 20:58:12.132217 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:12.132195    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k" event={"ID":"fc205950-caf0-4c47-84ce-a8fc54a1afb2","Type":"ContainerStarted","Data":"a824f0dcc8b6cb7447987e7dcb9f1e96271fb4720f64e0deeb3a3703602b918e"}
Sep 15 20:58:12.132453 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:12.132437    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:58:12.132604 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:12.132588    2564 scope.go:117] "RemoveContainer" containerID="47f607c932f9c83e32dbb2daf8205edf29d46d914b83dfd87714888296d8e55b"
Sep 15 20:58:12.148257 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:12.148238    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:58:13.138380 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:13.138179    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4c7k_fc205950-caf0-4c47-84ce-a8fc54a1afb2/ovn-acl-logging/0.log"
Sep 15 20:58:13.138829 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:13.138764    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k" event={"ID":"fc205950-caf0-4c47-84ce-a8fc54a1afb2","Type":"ContainerStarted","Data":"7151ff8fff34389de8044cc0e654acd178763b85633f5b2ac9a39ad98c92eb5a"}
Sep 15 20:58:13.138873 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:13.138850    2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 15 20:58:13.139125 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:13.139103    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:58:13.156195 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:13.156168    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:58:13.169363 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:13.169316    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k" podStartSLOduration=10.505715144 podStartE2EDuration="29.169300997s" totalImagesPullingTime="18.663585853s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:57:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:13.168083722 +0000 UTC m=+29.710246377" watchObservedRunningTime="2026-09-15 20:58:13.169300997 +0000 UTC m=+29.711463634"
Sep 15 20:58:13.422096 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:13.421996    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pjz7r"]
Sep 15 20:58:13.422228 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:13.422146    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:58:13.422293 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:13.422268    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjz7r" podUID="88840482-9724-4d2b-8343-3ed2ad33b258"
Sep 15 20:58:13.425226 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:13.425197    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m8fpw"]
Sep 15 20:58:13.425372 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:13.425310    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:58:13.425434 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:13.425385    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m8fpw" podUID="a70ba96c-ccb0-436a-aa3c-e056cb9a76e3"
Sep 15 20:58:13.428039 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:13.428012    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4dws2"]
Sep 15 20:58:13.428156 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:13.428111    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:58:13.428219 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:13.428193    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dws2" podUID="ce801675-8966-401c-b202-6631f49bdea0"
Sep 15 20:58:14.144143 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:14.144109    2564 generic.go:356] "Generic (PLEG): container finished" podID="b24f6335-59f4-4b7a-8548-56465ed60966" containerID="a116685faf7dd1de3c5835083cd58b2fe8447a560a44d093c4dd7b0d04039d40" exitCode=0
Sep 15 20:58:14.144526 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:14.144198    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8vrp" event={"ID":"b24f6335-59f4-4b7a-8548-56465ed60966","Type":"ContainerDied","Data":"a116685faf7dd1de3c5835083cd58b2fe8447a560a44d093c4dd7b0d04039d40"}
Sep 15 20:58:14.144526 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:14.144275    2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 15 20:58:14.275846 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:14.275821    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gwqhk"
Sep 15 20:58:14.276001 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:14.275967    2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 15 20:58:14.276528 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:14.276510    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gwqhk"
Sep 15 20:58:14.966388 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:14.966354    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:58:14.966388 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:14.966389    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:58:14.966645 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:14.966389    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:58:14.966645 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:14.966464    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4dws2" podUID="ce801675-8966-401c-b202-6631f49bdea0"
Sep 15 20:58:14.966645 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:14.966575    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pjz7r" podUID="88840482-9724-4d2b-8343-3ed2ad33b258"
Sep 15 20:58:14.966645 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:14.966635    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m8fpw" podUID="a70ba96c-ccb0-436a-aa3c-e056cb9a76e3"
Sep 15 20:58:15.146553 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:15.146516    2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 15 20:58:15.281497 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:15.281411    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:58:16.151522 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.151470    2564 generic.go:356] "Generic (PLEG): container finished" podID="b24f6335-59f4-4b7a-8548-56465ed60966" containerID="2d43d87d19e42755b582f280dd1da7e36aacb5a74d275bc78a016fc340dac832" exitCode=0
Sep 15 20:58:16.151952 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.151519    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8vrp" event={"ID":"b24f6335-59f4-4b7a-8548-56465ed60966","Type":"ContainerDied","Data":"2d43d87d19e42755b582f280dd1da7e36aacb5a74d275bc78a016fc340dac832"}
Sep 15 20:58:16.163455 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.163432    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4c7k"
Sep 15 20:58:16.751264 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.751235    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-221.ec2.internal" event="NodeReady"
Sep 15 20:58:16.751433 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.751396    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 15 20:58:16.783527 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.783331    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-dbwkr"]
Sep 15 20:58:16.787048 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.787025    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-dbwkr"
Sep 15 20:58:16.787688 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.787662    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-jsn2f"]
Sep 15 20:58:16.789914 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.789655    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-mbmjq\""
Sep 15 20:58:16.789914 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.789740    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 15 20:58:16.789914 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.789747    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 15 20:58:16.791554 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.791535    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-574cs"]
Sep 15 20:58:16.792017 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.791996    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jsn2f"
Sep 15 20:58:16.794702 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.794682    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 15 20:58:16.795178 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.795043    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 15 20:58:16.795178 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.795059    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-765nh\""
Sep 15 20:58:16.795326 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.795183    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 15 20:58:16.795739 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.795444    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-7t46m"]
Sep 15 20:58:16.795739 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.795545    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 15 20:58:16.795899 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.795803    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-574cs"
Sep 15 20:58:16.799775 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.799103    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 15 20:58:16.799775 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.799158    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 15 20:58:16.799775 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.799164    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-fh6zd\""
Sep 15 20:58:16.799775 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.799229    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-7t46m"
Sep 15 20:58:16.799775 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.799166    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ppbrv"]
Sep 15 20:58:16.799775 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.799679    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 15 20:58:16.802590 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.802570    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 15 20:58:16.802686 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.802603    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-846b7dcc97-fpgwt"]
Sep 15 20:58:16.802947 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.802918    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 15 20:58:16.802947 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.802922    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-nsxxs\""
Sep 15 20:58:16.803476 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.803213    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ppbrv"
Sep 15 20:58:16.804221 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.804093    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 15 20:58:16.804221 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.804098    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 15 20:58:16.806406 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.806384    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-dbwkr"]
Sep 15 20:58:16.806543 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.806475    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:16.807857 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.807838    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 15 20:58:16.807966 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.807902    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 15 20:58:16.808373 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.808357    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-jsn2f"]
Sep 15 20:58:16.810769 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.810748    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-lzdl4"]
Sep 15 20:58:16.812123 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.812105    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 15 20:58:16.814062 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.814039    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 15 20:58:16.814159 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.814102    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 15 20:58:16.814215 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.814154    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 15 20:58:16.814392 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.814373    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xpshw\""
Sep 15 20:58:16.814467 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.814401    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-t8cdw\""
Sep 15 20:58:16.814558 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.814380    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-7t46m"]
Sep 15 20:58:16.814673 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.814640    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-lzdl4"
Sep 15 20:58:16.815114 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.815099    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-574cs"]
Sep 15 20:58:16.815829 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.815809    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ppbrv"]
Sep 15 20:58:16.818933 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.818693    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 15 20:58:16.819709 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.819691    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 15 20:58:16.822463 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.822432    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 15 20:58:16.822805 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.822786    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 15 20:58:16.829571 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.826255    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-pghnz\""
Sep 15 20:58:16.833006 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.832558    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 15 20:58:16.833171 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.833151    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-846b7dcc97-fpgwt"]
Sep 15 20:58:16.834547 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.834530    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-lzdl4"]
Sep 15 20:58:16.834708 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.834680    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5mvsw"]
Sep 15 20:58:16.838234 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.838218    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rqmrf"]
Sep 15 20:58:16.838452 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.838434    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5mvsw"
Sep 15 20:58:16.840911 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.840892    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 15 20:58:16.841016 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.840892    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 15 20:58:16.841202 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.841187    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 15 20:58:16.841296 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.841266    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wvdjj\""
Sep 15 20:58:16.841705 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.841687    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rqmrf"
Sep 15 20:58:16.843856 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.843837    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p626r\""
Sep 15 20:58:16.844339 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.844296    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 15 20:58:16.844547 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.844530    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 15 20:58:16.845381 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.845360    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5mvsw"]
Sep 15 20:58:16.847988 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.847968    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rqmrf"]
Sep 15 20:58:16.936242 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936149    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f3b99f56-873f-40d8-abf3-b0c02523b2f7-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ppbrv\" (UID: \"f3b99f56-873f-40d8-abf3-b0c02523b2f7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ppbrv"
Sep 15 20:58:16.936242 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936182    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dcfdcceb-cea6-4f5a-958f-a1b558bc1a1f-serving-cert\") pod \"service-ca-operator-cf7c776b7-jsn2f\" (UID: \"dcfdcceb-cea6-4f5a-958f-a1b558bc1a1f\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jsn2f"
Sep 15 20:58:16.936242 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936200    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf-config\") pod \"console-operator-696884fb9c-7t46m\" (UID: \"c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf\") " pod="openshift-console-operator/console-operator-696884fb9c-7t46m"
Sep 15 20:58:16.936242 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936217    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/04865999-e56e-4a65-9c4a-b52647e7b1db-config-volume\") pod \"dns-default-rqmrf\" (UID: \"04865999-e56e-4a65-9c4a-b52647e7b1db\") " pod="openshift-dns/dns-default-rqmrf"
Sep 15 20:58:16.936242 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936236    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/04865999-e56e-4a65-9c4a-b52647e7b1db-metrics-tls\") pod \"dns-default-rqmrf\" (UID: \"04865999-e56e-4a65-9c4a-b52647e7b1db\") " pod="openshift-dns/dns-default-rqmrf"
Sep 15 20:58:16.936536 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936282    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk5md\" (UniqueName: \"kubernetes.io/projected/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-kube-api-access-rk5md\") pod \"cluster-samples-operator-86499c89c4-574cs\" (UID: \"1d7ebda7-74f1-4762-b7a9-78008fa7f9cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-574cs"
Sep 15 20:58:16.936536 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936322    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2243bd69-2640-4dc6-beaa-7e22b414db60-trusted-ca\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:16.936536 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936362    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-bound-sa-token\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:16.936536 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936381    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc5nt\" (UniqueName: \"kubernetes.io/projected/04865999-e56e-4a65-9c4a-b52647e7b1db-kube-api-access-lc5nt\") pod \"dns-default-rqmrf\" (UID: \"04865999-e56e-4a65-9c4a-b52647e7b1db\") " pod="openshift-dns/dns-default-rqmrf"
Sep 15 20:58:16.936536 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936425    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8blvv\" (UniqueName: \"kubernetes.io/projected/dcfdcceb-cea6-4f5a-958f-a1b558bc1a1f-kube-api-access-8blvv\") pod \"service-ca-operator-cf7c776b7-jsn2f\" (UID: \"dcfdcceb-cea6-4f5a-958f-a1b558bc1a1f\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jsn2f"
Sep 15 20:58:16.936536 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936450    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2243bd69-2640-4dc6-beaa-7e22b414db60-installation-pull-secrets\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:16.936536 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936465    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04865999-e56e-4a65-9c4a-b52647e7b1db-tmp-dir\") pod \"dns-default-rqmrf\" (UID: \"04865999-e56e-4a65-9c4a-b52647e7b1db\") " pod="openshift-dns/dns-default-rqmrf"
Sep 15 20:58:16.936536 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936502    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f3b99f56-873f-40d8-abf3-b0c02523b2f7-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ppbrv\" (UID: \"f3b99f56-873f-40d8-abf3-b0c02523b2f7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ppbrv"
Sep 15 20:58:16.936930 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936562    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2aae8637-fdbb-4c3c-a878-ec6ea9a25c7b-cert\") pod \"ingress-canary-5mvsw\" (UID: \"2aae8637-fdbb-4c3c-a878-ec6ea9a25c7b\") " pod="openshift-ingress-canary/ingress-canary-5mvsw"
Sep 15 20:58:16.936930 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936605    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf-trusted-ca\") pod \"console-operator-696884fb9c-7t46m\" (UID: \"c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf\") " pod="openshift-console-operator/console-operator-696884fb9c-7t46m"
Sep 15 20:58:16.936930 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936635    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2243bd69-2640-4dc6-beaa-7e22b414db60-image-registry-private-configuration\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:16.936930 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936667    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf-serving-cert\") pod \"console-operator-696884fb9c-7t46m\" (UID: \"c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf\") " pod="openshift-console-operator/console-operator-696884fb9c-7t46m"
Sep 15 20:58:16.936930 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936691    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2243bd69-2640-4dc6-beaa-7e22b414db60-ca-trust-extracted\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:16.936930 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936717    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72wcj\" (UniqueName: \"kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-kube-api-access-72wcj\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:16.936930 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936788    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dcfdcceb-cea6-4f5a-958f-a1b558bc1a1f-config\") pod \"service-ca-operator-cf7c776b7-jsn2f\" (UID: \"dcfdcceb-cea6-4f5a-958f-a1b558bc1a1f\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jsn2f"
Sep 15 20:58:16.936930 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936813    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-574cs\" (UID: \"1d7ebda7-74f1-4762-b7a9-78008fa7f9cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-574cs"
Sep 15 20:58:16.936930 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936844    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n2zq\" (UniqueName: \"kubernetes.io/projected/26e0bf54-90c8-4daa-9808-16b7501bc0a3-kube-api-access-7n2zq\") pod \"volume-data-source-validator-86695b7cc7-dbwkr\" (UID: \"26e0bf54-90c8-4daa-9808-16b7501bc0a3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-dbwkr"
Sep 15 20:58:16.936930 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936878    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcxxz\" (UniqueName: \"kubernetes.io/projected/2aae8637-fdbb-4c3c-a878-ec6ea9a25c7b-kube-api-access-pcxxz\") pod \"ingress-canary-5mvsw\" (UID: \"2aae8637-fdbb-4c3c-a878-ec6ea9a25c7b\") " pod="openshift-ingress-canary/ingress-canary-5mvsw"
Sep 15 20:58:16.936930 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936906    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zdz5\" (UniqueName: \"kubernetes.io/projected/f3b99f56-873f-40d8-abf3-b0c02523b2f7-kube-api-access-5zdz5\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ppbrv\" (UID: \"f3b99f56-873f-40d8-abf3-b0c02523b2f7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ppbrv"
Sep 15 20:58:16.936930 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936931    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdsrz\" (UniqueName: \"kubernetes.io/projected/c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf-kube-api-access-kdsrz\") pod \"console-operator-696884fb9c-7t46m\" (UID: \"c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf\") " pod="openshift-console-operator/console-operator-696884fb9c-7t46m"
Sep 15 20:58:16.937332 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936954    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:16.937332 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.936983    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-certificates\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:16.937332 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.937009    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6lmq\" (UniqueName: \"kubernetes.io/projected/90967cfe-f101-4cf3-bdc5-b0e156f578c1-kube-api-access-h6lmq\") pod \"network-check-source-6f8b768455-lzdl4\" (UID: \"90967cfe-f101-4cf3-bdc5-b0e156f578c1\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-lzdl4"
Sep 15 20:58:16.963087 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.963052    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:16.966838 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.966808    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:58:16.966985 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.966812    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:58:16.967049 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.966816    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:58:16.969507 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.969470    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 15 20:58:16.969702 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.969686    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 15 20:58:16.969883 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.969868    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-js7d5\""
Sep 15 20:58:16.970056 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:16.970042    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-r6tvw\""
Sep 15 20:58:17.038341 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038310    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f3b99f56-873f-40d8-abf3-b0c02523b2f7-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ppbrv\" (UID: \"f3b99f56-873f-40d8-abf3-b0c02523b2f7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ppbrv"
Sep 15 20:58:17.038563 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038346    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dcfdcceb-cea6-4f5a-958f-a1b558bc1a1f-serving-cert\") pod \"service-ca-operator-cf7c776b7-jsn2f\" (UID: \"dcfdcceb-cea6-4f5a-958f-a1b558bc1a1f\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jsn2f"
Sep 15 20:58:17.038563 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038374    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf-config\") pod \"console-operator-696884fb9c-7t46m\" (UID: \"c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf\") " pod="openshift-console-operator/console-operator-696884fb9c-7t46m"
Sep 15 20:58:17.038563 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038418    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/04865999-e56e-4a65-9c4a-b52647e7b1db-config-volume\") pod \"dns-default-rqmrf\" (UID: \"04865999-e56e-4a65-9c4a-b52647e7b1db\") " pod="openshift-dns/dns-default-rqmrf"
Sep 15 20:58:17.038563 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038440    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/04865999-e56e-4a65-9c4a-b52647e7b1db-metrics-tls\") pod \"dns-default-rqmrf\" (UID: \"04865999-e56e-4a65-9c4a-b52647e7b1db\") " pod="openshift-dns/dns-default-rqmrf"
Sep 15 20:58:17.038563 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038464    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk5md\" (UniqueName: \"kubernetes.io/projected/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-kube-api-access-rk5md\") pod \"cluster-samples-operator-86499c89c4-574cs\" (UID: \"1d7ebda7-74f1-4762-b7a9-78008fa7f9cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-574cs"
Sep 15 20:58:17.038563 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038508    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2243bd69-2640-4dc6-beaa-7e22b414db60-trusted-ca\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:17.038563 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038535    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-bound-sa-token\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:17.038563 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038555    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc5nt\" (UniqueName: \"kubernetes.io/projected/04865999-e56e-4a65-9c4a-b52647e7b1db-kube-api-access-lc5nt\") pod \"dns-default-rqmrf\" (UID: \"04865999-e56e-4a65-9c4a-b52647e7b1db\") " pod="openshift-dns/dns-default-rqmrf"
Sep 15 20:58:17.038910 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038591    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8blvv\" (UniqueName: \"kubernetes.io/projected/dcfdcceb-cea6-4f5a-958f-a1b558bc1a1f-kube-api-access-8blvv\") pod \"service-ca-operator-cf7c776b7-jsn2f\" (UID: \"dcfdcceb-cea6-4f5a-958f-a1b558bc1a1f\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jsn2f"
Sep 15 20:58:17.038910 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038617    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2243bd69-2640-4dc6-beaa-7e22b414db60-installation-pull-secrets\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:17.038910 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038639    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04865999-e56e-4a65-9c4a-b52647e7b1db-tmp-dir\") pod \"dns-default-rqmrf\" (UID: \"04865999-e56e-4a65-9c4a-b52647e7b1db\") " pod="openshift-dns/dns-default-rqmrf"
Sep 15 20:58:17.038910 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038681    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f3b99f56-873f-40d8-abf3-b0c02523b2f7-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ppbrv\" (UID: \"f3b99f56-873f-40d8-abf3-b0c02523b2f7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ppbrv"
Sep 15 20:58:17.038910 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038705    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2aae8637-fdbb-4c3c-a878-ec6ea9a25c7b-cert\") pod \"ingress-canary-5mvsw\" (UID: \"2aae8637-fdbb-4c3c-a878-ec6ea9a25c7b\") " pod="openshift-ingress-canary/ingress-canary-5mvsw"
Sep 15 20:58:17.038910 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038725    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf-trusted-ca\") pod \"console-operator-696884fb9c-7t46m\" (UID: \"c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf\") " pod="openshift-console-operator/console-operator-696884fb9c-7t46m"
Sep 15 20:58:17.038910 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038745    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2243bd69-2640-4dc6-beaa-7e22b414db60-image-registry-private-configuration\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:17.038910 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038769    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf-serving-cert\") pod \"console-operator-696884fb9c-7t46m\" (UID: \"c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf\") " pod="openshift-console-operator/console-operator-696884fb9c-7t46m"
Sep 15 20:58:17.038910 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038790    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2243bd69-2640-4dc6-beaa-7e22b414db60-ca-trust-extracted\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:17.038910 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038811    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72wcj\" (UniqueName: \"kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-kube-api-access-72wcj\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:17.038910 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038851    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dcfdcceb-cea6-4f5a-958f-a1b558bc1a1f-config\") pod \"service-ca-operator-cf7c776b7-jsn2f\" (UID: \"dcfdcceb-cea6-4f5a-958f-a1b558bc1a1f\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jsn2f"
Sep 15 20:58:17.038910 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038896    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-574cs\" (UID: \"1d7ebda7-74f1-4762-b7a9-78008fa7f9cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-574cs"
Sep 15 20:58:17.039443 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038925    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n2zq\" (UniqueName: \"kubernetes.io/projected/26e0bf54-90c8-4daa-9808-16b7501bc0a3-kube-api-access-7n2zq\") pod \"volume-data-source-validator-86695b7cc7-dbwkr\" (UID: \"26e0bf54-90c8-4daa-9808-16b7501bc0a3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-dbwkr"
Sep 15 20:58:17.039443 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038946    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcxxz\" (UniqueName: \"kubernetes.io/projected/2aae8637-fdbb-4c3c-a878-ec6ea9a25c7b-kube-api-access-pcxxz\") pod \"ingress-canary-5mvsw\" (UID: \"2aae8637-fdbb-4c3c-a878-ec6ea9a25c7b\") " pod="openshift-ingress-canary/ingress-canary-5mvsw"
Sep 15 20:58:17.039443 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038977    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zdz5\" (UniqueName: \"kubernetes.io/projected/f3b99f56-873f-40d8-abf3-b0c02523b2f7-kube-api-access-5zdz5\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ppbrv\" (UID: \"f3b99f56-873f-40d8-abf3-b0c02523b2f7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ppbrv"
Sep 15 20:58:17.039443 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.038999    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdsrz\" (UniqueName: \"kubernetes.io/projected/c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf-kube-api-access-kdsrz\") pod \"console-operator-696884fb9c-7t46m\" (UID: \"c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf\") " pod="openshift-console-operator/console-operator-696884fb9c-7t46m"
Sep 15 20:58:17.039443 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.039018    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:17.039443 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.039036    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf-config\") pod \"console-operator-696884fb9c-7t46m\" (UID: \"c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf\") " pod="openshift-console-operator/console-operator-696884fb9c-7t46m"
Sep 15 20:58:17.039443 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.039043    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-certificates\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:17.039443 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.039088    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6lmq\" (UniqueName: \"kubernetes.io/projected/90967cfe-f101-4cf3-bdc5-b0e156f578c1-kube-api-access-h6lmq\") pod \"network-check-source-6f8b768455-lzdl4\" (UID: \"90967cfe-f101-4cf3-bdc5-b0e156f578c1\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-lzdl4"
Sep 15 20:58:17.039443 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:17.039393    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:58:17.040017 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:17.039468    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04865999-e56e-4a65-9c4a-b52647e7b1db-metrics-tls podName:04865999-e56e-4a65-9c4a-b52647e7b1db nodeName:}" failed. No retries permitted until 2026-09-15 20:58:17.539447938 +0000 UTC m=+34.081610565 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/04865999-e56e-4a65-9c4a-b52647e7b1db-metrics-tls") pod "dns-default-rqmrf" (UID: "04865999-e56e-4a65-9c4a-b52647e7b1db") : secret "dns-default-metrics-tls" not found
Sep 15 20:58:17.040017 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:17.039588    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 15 20:58:17.040017 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.039591    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-certificates\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:17.040017 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:17.039642    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-samples-operator-tls podName:1d7ebda7-74f1-4762-b7a9-78008fa7f9cc nodeName:}" failed. No retries permitted until 2026-09-15 20:58:17.539628664 +0000 UTC m=+34.081791298 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-574cs" (UID: "1d7ebda7-74f1-4762-b7a9-78008fa7f9cc") : secret "samples-operator-tls" not found
Sep 15 20:58:17.040017 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.039796    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/04865999-e56e-4a65-9c4a-b52647e7b1db-config-volume\") pod \"dns-default-rqmrf\" (UID: \"04865999-e56e-4a65-9c4a-b52647e7b1db\") " pod="openshift-dns/dns-default-rqmrf"
Sep 15 20:58:17.040017 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.039904    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf-trusted-ca\") pod \"console-operator-696884fb9c-7t46m\" (UID: \"c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf\") " pod="openshift-console-operator/console-operator-696884fb9c-7t46m"
Sep 15 20:58:17.040017 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.039995    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2243bd69-2640-4dc6-beaa-7e22b414db60-ca-trust-extracted\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:17.040407 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.040255    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2243bd69-2640-4dc6-beaa-7e22b414db60-trusted-ca\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:17.040407 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:17.040282    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 20:58:17.040407 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:17.040298    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-846b7dcc97-fpgwt: secret "image-registry-tls" not found
Sep 15 20:58:17.040407 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:17.040344    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls podName:2243bd69-2640-4dc6-beaa-7e22b414db60 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:17.540328188 +0000 UTC m=+34.082490815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls") pod "image-registry-846b7dcc97-fpgwt" (UID: "2243bd69-2640-4dc6-beaa-7e22b414db60") : secret "image-registry-tls" not found
Sep 15 20:58:17.040745 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.040717    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f3b99f56-873f-40d8-abf3-b0c02523b2f7-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ppbrv\" (UID: \"f3b99f56-873f-40d8-abf3-b0c02523b2f7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ppbrv"
Sep 15 20:58:17.040816 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.040756    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dcfdcceb-cea6-4f5a-958f-a1b558bc1a1f-config\") pod \"service-ca-operator-cf7c776b7-jsn2f\" (UID: \"dcfdcceb-cea6-4f5a-958f-a1b558bc1a1f\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jsn2f"
Sep 15 20:58:17.040998 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.040975    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04865999-e56e-4a65-9c4a-b52647e7b1db-tmp-dir\") pod \"dns-default-rqmrf\" (UID: \"04865999-e56e-4a65-9c4a-b52647e7b1db\") " pod="openshift-dns/dns-default-rqmrf"
Sep 15 20:58:17.044024 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.044001    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f3b99f56-873f-40d8-abf3-b0c02523b2f7-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ppbrv\" (UID: \"f3b99f56-873f-40d8-abf3-b0c02523b2f7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ppbrv"
Sep 15 20:58:17.044131 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.044013    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf-serving-cert\") pod \"console-operator-696884fb9c-7t46m\" (UID: \"c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf\") " pod="openshift-console-operator/console-operator-696884fb9c-7t46m"
Sep 15 20:58:17.044192 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.044131    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dcfdcceb-cea6-4f5a-958f-a1b558bc1a1f-serving-cert\") pod \"service-ca-operator-cf7c776b7-jsn2f\" (UID: \"dcfdcceb-cea6-4f5a-958f-a1b558bc1a1f\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jsn2f"
Sep 15 20:58:17.044246 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.044218    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2243bd69-2640-4dc6-beaa-7e22b414db60-image-registry-private-configuration\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:17.044302 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.044276    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2aae8637-fdbb-4c3c-a878-ec6ea9a25c7b-cert\") pod \"ingress-canary-5mvsw\" (UID: \"2aae8637-fdbb-4c3c-a878-ec6ea9a25c7b\") " pod="openshift-ingress-canary/ingress-canary-5mvsw"
Sep 15 20:58:17.045242 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.045220    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2243bd69-2640-4dc6-beaa-7e22b414db60-installation-pull-secrets\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:17.054253 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.054159    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8blvv\" (UniqueName: \"kubernetes.io/projected/dcfdcceb-cea6-4f5a-958f-a1b558bc1a1f-kube-api-access-8blvv\") pod \"service-ca-operator-cf7c776b7-jsn2f\" (UID: \"dcfdcceb-cea6-4f5a-958f-a1b558bc1a1f\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jsn2f"
Sep 15 20:58:17.054968 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.054912    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:17.055113 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.055079    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72wcj\" (UniqueName: \"kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-kube-api-access-72wcj\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:17.057048 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.056601    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc5nt\" (UniqueName: \"kubernetes.io/projected/04865999-e56e-4a65-9c4a-b52647e7b1db-kube-api-access-lc5nt\") pod \"dns-default-rqmrf\" (UID: \"04865999-e56e-4a65-9c4a-b52647e7b1db\") " pod="openshift-dns/dns-default-rqmrf"
Sep 15 20:58:17.057048 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.056809    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk5md\" (UniqueName: \"kubernetes.io/projected/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-kube-api-access-rk5md\") pod \"cluster-samples-operator-86499c89c4-574cs\" (UID: \"1d7ebda7-74f1-4762-b7a9-78008fa7f9cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-574cs"
Sep 15 20:58:17.057048 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.056988    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6lmq\" (UniqueName: \"kubernetes.io/projected/90967cfe-f101-4cf3-bdc5-b0e156f578c1-kube-api-access-h6lmq\") pod \"network-check-source-6f8b768455-lzdl4\" (UID: \"90967cfe-f101-4cf3-bdc5-b0e156f578c1\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-lzdl4"
Sep 15 20:58:17.057048 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.056999    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zdz5\" (UniqueName: \"kubernetes.io/projected/f3b99f56-873f-40d8-abf3-b0c02523b2f7-kube-api-access-5zdz5\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ppbrv\" (UID: \"f3b99f56-873f-40d8-abf3-b0c02523b2f7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ppbrv"
Sep 15 20:58:17.057291 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.057160    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-bound-sa-token\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:17.057383 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.057358    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcxxz\" (UniqueName: \"kubernetes.io/projected/2aae8637-fdbb-4c3c-a878-ec6ea9a25c7b-kube-api-access-pcxxz\") pod \"ingress-canary-5mvsw\" (UID: \"2aae8637-fdbb-4c3c-a878-ec6ea9a25c7b\") " pod="openshift-ingress-canary/ingress-canary-5mvsw"
Sep 15 20:58:17.058207 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.058183    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n2zq\" (UniqueName: \"kubernetes.io/projected/26e0bf54-90c8-4daa-9808-16b7501bc0a3-kube-api-access-7n2zq\") pod \"volume-data-source-validator-86695b7cc7-dbwkr\" (UID: \"26e0bf54-90c8-4daa-9808-16b7501bc0a3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-dbwkr"
Sep 15 20:58:17.058909 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.058885    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdsrz\" (UniqueName: \"kubernetes.io/projected/c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf-kube-api-access-kdsrz\") pod \"console-operator-696884fb9c-7t46m\" (UID: \"c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf\") " pod="openshift-console-operator/console-operator-696884fb9c-7t46m"
Sep 15 20:58:17.103023 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.102989    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-dbwkr"
Sep 15 20:58:17.111229 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.111202    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jsn2f"
Sep 15 20:58:17.132528 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.132479    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-7t46m"
Sep 15 20:58:17.140327 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.140301    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ppbrv"
Sep 15 20:58:17.157135 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.156921    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-lzdl4"
Sep 15 20:58:17.165578 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.165365    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5mvsw"
Sep 15 20:58:17.318026 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.317987    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-2lm2s"]
Sep 15 20:58:17.323836 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.323798    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2lm2s"
Sep 15 20:58:17.326459 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.326433    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 15 20:58:17.326903 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.326879    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 15 20:58:17.327004 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.326879    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-wlh8s\""
Sep 15 20:58:17.330578 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.330556    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-2lm2s"]
Sep 15 20:58:17.403307 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.403275    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-dbwkr"]
Sep 15 20:58:17.407123 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.407096    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-jsn2f"]
Sep 15 20:58:17.407999 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:58:17.407970    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26e0bf54_90c8_4daa_9808_16b7501bc0a3.slice/crio-fbc009c93a23b1d502e0ea5ba8749a9d5d2b2277d044816a1c3099002a78a8b6 WatchSource:0}: Error finding container fbc009c93a23b1d502e0ea5ba8749a9d5d2b2277d044816a1c3099002a78a8b6: Status 404 returned error can't find the container with id fbc009c93a23b1d502e0ea5ba8749a9d5d2b2277d044816a1c3099002a78a8b6
Sep 15 20:58:17.408866 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:58:17.408833    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcfdcceb_cea6_4f5a_958f_a1b558bc1a1f.slice/crio-d0d90724f04e8c0e92a99e5903506bf73bb6670798f63776097ca7a4fded8ea8 WatchSource:0}: Error finding container d0d90724f04e8c0e92a99e5903506bf73bb6670798f63776097ca7a4fded8ea8: Status 404 returned error can't find the container with id d0d90724f04e8c0e92a99e5903506bf73bb6670798f63776097ca7a4fded8ea8
Sep 15 20:58:17.431972 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.431937    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-7t46m"]
Sep 15 20:58:17.435809 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:58:17.435778    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc41a4ff2_5904_45fc_89fd_7f8cbd0a9dcf.slice/crio-785843cebc14070b1ae1016eacb7a32cff863d370a8218ccbe4048897a36e863 WatchSource:0}: Error finding container 785843cebc14070b1ae1016eacb7a32cff863d370a8218ccbe4048897a36e863: Status 404 returned error can't find the container with id 785843cebc14070b1ae1016eacb7a32cff863d370a8218ccbe4048897a36e863
Sep 15 20:58:17.443157 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.443072    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2lm2s\" (UID: \"ac0aff21-ec17-47fe-b4a2-b90b5d07786b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2lm2s"
Sep 15 20:58:17.443157 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.443134    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-2lm2s\" (UID: \"ac0aff21-ec17-47fe-b4a2-b90b5d07786b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2lm2s"
Sep 15 20:58:17.460559 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.460438    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ppbrv"]
Sep 15 20:58:17.463753 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:58:17.463724    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3b99f56_873f_40d8_abf3_b0c02523b2f7.slice/crio-c278f9ad62f03635080ae22861fb88c2475f1899dec6a741dff77aa9d0014259 WatchSource:0}: Error finding container c278f9ad62f03635080ae22861fb88c2475f1899dec6a741dff77aa9d0014259: Status 404 returned error can't find the container with id c278f9ad62f03635080ae22861fb88c2475f1899dec6a741dff77aa9d0014259
Sep 15 20:58:17.463957 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.463934    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-lzdl4"]
Sep 15 20:58:17.465659 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:58:17.465637    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90967cfe_f101_4cf3_bdc5_b0e156f578c1.slice/crio-a36686f19fc07461dc86819964eb1ac89523d8728366d580423a762bfe5b816e WatchSource:0}: Error finding container a36686f19fc07461dc86819964eb1ac89523d8728366d580423a762bfe5b816e: Status 404 returned error can't find the container with id a36686f19fc07461dc86819964eb1ac89523d8728366d580423a762bfe5b816e
Sep 15 20:58:17.489596 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.489560    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5mvsw"]
Sep 15 20:58:17.493120 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:58:17.493092    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2aae8637_fdbb_4c3c_a878_ec6ea9a25c7b.slice/crio-bf3e426e469d90d82565e69df5e20e7810ecd3bc4faa81f5f5b990d4d199d6bc WatchSource:0}: Error finding container bf3e426e469d90d82565e69df5e20e7810ecd3bc4faa81f5f5b990d4d199d6bc: Status 404 returned error can't find the container with id bf3e426e469d90d82565e69df5e20e7810ecd3bc4faa81f5f5b990d4d199d6bc
Sep 15 20:58:17.544251 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.544218    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2lm2s\" (UID: \"ac0aff21-ec17-47fe-b4a2-b90b5d07786b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2lm2s"
Sep 15 20:58:17.544436 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.544276    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-2lm2s\" (UID: \"ac0aff21-ec17-47fe-b4a2-b90b5d07786b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2lm2s"
Sep 15 20:58:17.544436 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.544316    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-574cs\" (UID: \"1d7ebda7-74f1-4762-b7a9-78008fa7f9cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-574cs"
Sep 15 20:58:17.544436 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.544347    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:17.544436 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:17.544375    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 15 20:58:17.544647 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:17.544450    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-networking-console-plugin-cert podName:ac0aff21-ec17-47fe-b4a2-b90b5d07786b nodeName:}" failed. No retries permitted until 2026-09-15 20:58:18.044429529 +0000 UTC m=+34.586592157 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-2lm2s" (UID: "ac0aff21-ec17-47fe-b4a2-b90b5d07786b") : secret "networking-console-plugin-cert" not found
Sep 15 20:58:17.544647 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:17.544471    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 15 20:58:17.544647 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.544383    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/04865999-e56e-4a65-9c4a-b52647e7b1db-metrics-tls\") pod \"dns-default-rqmrf\" (UID: \"04865999-e56e-4a65-9c4a-b52647e7b1db\") " pod="openshift-dns/dns-default-rqmrf"
Sep 15 20:58:17.544647 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:17.544471    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:58:17.544647 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:17.544556    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-samples-operator-tls podName:1d7ebda7-74f1-4762-b7a9-78008fa7f9cc nodeName:}" failed. No retries permitted until 2026-09-15 20:58:18.544537783 +0000 UTC m=+35.086700397 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-574cs" (UID: "1d7ebda7-74f1-4762-b7a9-78008fa7f9cc") : secret "samples-operator-tls" not found
Sep 15 20:58:17.544647 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:17.544475    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 20:58:17.544647 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:17.544576    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-846b7dcc97-fpgwt: secret "image-registry-tls" not found
Sep 15 20:58:17.544647 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:17.544601    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04865999-e56e-4a65-9c4a-b52647e7b1db-metrics-tls podName:04865999-e56e-4a65-9c4a-b52647e7b1db nodeName:}" failed. No retries permitted until 2026-09-15 20:58:18.544575007 +0000 UTC m=+35.086737631 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/04865999-e56e-4a65-9c4a-b52647e7b1db-metrics-tls") pod "dns-default-rqmrf" (UID: "04865999-e56e-4a65-9c4a-b52647e7b1db") : secret "dns-default-metrics-tls" not found
Sep 15 20:58:17.544647 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:17.544627    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls podName:2243bd69-2640-4dc6-beaa-7e22b414db60 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:18.544619324 +0000 UTC m=+35.086781936 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls") pod "image-registry-846b7dcc97-fpgwt" (UID: "2243bd69-2640-4dc6-beaa-7e22b414db60") : secret "image-registry-tls" not found
Sep 15 20:58:17.545078 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.545060    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-2lm2s\" (UID: \"ac0aff21-ec17-47fe-b4a2-b90b5d07786b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2lm2s"
Sep 15 20:58:17.645276 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.645240    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs\") pod \"network-metrics-daemon-4dws2\" (UID: \"ce801675-8966-401c-b202-6631f49bdea0\") " pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:58:17.645432 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:17.645408    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 15 20:58:17.645514 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:17.645502    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs podName:ce801675-8966-401c-b202-6631f49bdea0 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:49.645469644 +0000 UTC m=+66.187632260 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs") pod "network-metrics-daemon-4dws2" (UID: "ce801675-8966-401c-b202-6631f49bdea0") : secret "metrics-daemon-secret" not found
Sep 15 20:58:17.846890 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.846843    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88840482-9724-4d2b-8343-3ed2ad33b258-original-pull-secret\") pod \"global-pull-secret-syncer-pjz7r\" (UID: \"88840482-9724-4d2b-8343-3ed2ad33b258\") " pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:58:17.847083 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.846958    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcpx9\" (UniqueName: \"kubernetes.io/projected/a70ba96c-ccb0-436a-aa3c-e056cb9a76e3-kube-api-access-dcpx9\") pod \"network-check-target-m8fpw\" (UID: \"a70ba96c-ccb0-436a-aa3c-e056cb9a76e3\") " pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:58:17.850104 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.850078    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88840482-9724-4d2b-8343-3ed2ad33b258-original-pull-secret\") pod \"global-pull-secret-syncer-pjz7r\" (UID: \"88840482-9724-4d2b-8343-3ed2ad33b258\") " pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:58:17.850316 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.850293    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcpx9\" (UniqueName: \"kubernetes.io/projected/a70ba96c-ccb0-436a-aa3c-e056cb9a76e3-kube-api-access-dcpx9\") pod \"network-check-target-m8fpw\" (UID: \"a70ba96c-ccb0-436a-aa3c-e056cb9a76e3\") " pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:58:17.879242 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.879199    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pjz7r"
Sep 15 20:58:17.885619 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:17.885599    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:58:18.049656 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:18.049299    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2lm2s\" (UID: \"ac0aff21-ec17-47fe-b4a2-b90b5d07786b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2lm2s"
Sep 15 20:58:18.049656 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:18.049580    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 15 20:58:18.049895 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:18.049663    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-networking-console-plugin-cert podName:ac0aff21-ec17-47fe-b4a2-b90b5d07786b nodeName:}" failed. No retries permitted until 2026-09-15 20:58:19.049641742 +0000 UTC m=+35.591804371 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-2lm2s" (UID: "ac0aff21-ec17-47fe-b4a2-b90b5d07786b") : secret "networking-console-plugin-cert" not found
Sep 15 20:58:18.159048 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:18.158979    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pjz7r"]
Sep 15 20:58:18.166827 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:18.166797    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m8fpw"]
Sep 15 20:58:18.172013 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:18.171838    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5mvsw" event={"ID":"2aae8637-fdbb-4c3c-a878-ec6ea9a25c7b","Type":"ContainerStarted","Data":"bf3e426e469d90d82565e69df5e20e7810ecd3bc4faa81f5f5b990d4d199d6bc"}
Sep 15 20:58:18.178044 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:18.178008    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-dbwkr" event={"ID":"26e0bf54-90c8-4daa-9808-16b7501bc0a3","Type":"ContainerStarted","Data":"fbc009c93a23b1d502e0ea5ba8749a9d5d2b2277d044816a1c3099002a78a8b6"}
Sep 15 20:58:18.189448 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:18.187522    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-lzdl4" event={"ID":"90967cfe-f101-4cf3-bdc5-b0e156f578c1","Type":"ContainerStarted","Data":"a36686f19fc07461dc86819964eb1ac89523d8728366d580423a762bfe5b816e"}
Sep 15 20:58:18.194226 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:18.194133    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jsn2f" event={"ID":"dcfdcceb-cea6-4f5a-958f-a1b558bc1a1f","Type":"ContainerStarted","Data":"d0d90724f04e8c0e92a99e5903506bf73bb6670798f63776097ca7a4fded8ea8"}
Sep 15 20:58:18.196707 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:18.196623    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ppbrv" event={"ID":"f3b99f56-873f-40d8-abf3-b0c02523b2f7","Type":"ContainerStarted","Data":"c278f9ad62f03635080ae22861fb88c2475f1899dec6a741dff77aa9d0014259"}
Sep 15 20:58:18.206175 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:18.206075    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-7t46m" event={"ID":"c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf","Type":"ContainerStarted","Data":"785843cebc14070b1ae1016eacb7a32cff863d370a8218ccbe4048897a36e863"}
Sep 15 20:58:18.557729 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:18.556750    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-574cs\" (UID: \"1d7ebda7-74f1-4762-b7a9-78008fa7f9cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-574cs"
Sep 15 20:58:18.557729 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:18.556811    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:18.557729 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:18.556857    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/04865999-e56e-4a65-9c4a-b52647e7b1db-metrics-tls\") pod \"dns-default-rqmrf\" (UID: \"04865999-e56e-4a65-9c4a-b52647e7b1db\") " pod="openshift-dns/dns-default-rqmrf"
Sep 15 20:58:18.557729 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:18.556990    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:58:18.557729 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:18.557056    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04865999-e56e-4a65-9c4a-b52647e7b1db-metrics-tls podName:04865999-e56e-4a65-9c4a-b52647e7b1db nodeName:}" failed. No retries permitted until 2026-09-15 20:58:20.557035579 +0000 UTC m=+37.099198212 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/04865999-e56e-4a65-9c4a-b52647e7b1db-metrics-tls") pod "dns-default-rqmrf" (UID: "04865999-e56e-4a65-9c4a-b52647e7b1db") : secret "dns-default-metrics-tls" not found
Sep 15 20:58:18.557729 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:18.557468    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 15 20:58:18.557729 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:18.557557    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-samples-operator-tls podName:1d7ebda7-74f1-4762-b7a9-78008fa7f9cc nodeName:}" failed. No retries permitted until 2026-09-15 20:58:20.557528373 +0000 UTC m=+37.099691003 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-574cs" (UID: "1d7ebda7-74f1-4762-b7a9-78008fa7f9cc") : secret "samples-operator-tls" not found
Sep 15 20:58:18.557729 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:18.557636    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 20:58:18.557729 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:18.557648    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-846b7dcc97-fpgwt: secret "image-registry-tls" not found
Sep 15 20:58:18.557729 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:18.557683    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls podName:2243bd69-2640-4dc6-beaa-7e22b414db60 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:20.557672449 +0000 UTC m=+37.099835076 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls") pod "image-registry-846b7dcc97-fpgwt" (UID: "2243bd69-2640-4dc6-beaa-7e22b414db60") : secret "image-registry-tls" not found
Sep 15 20:58:19.062389 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:19.061757    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2lm2s\" (UID: \"ac0aff21-ec17-47fe-b4a2-b90b5d07786b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2lm2s"
Sep 15 20:58:19.062389 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:19.061959    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 15 20:58:19.062389 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:19.062017    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-networking-console-plugin-cert podName:ac0aff21-ec17-47fe-b4a2-b90b5d07786b nodeName:}" failed. No retries permitted until 2026-09-15 20:58:21.061999993 +0000 UTC m=+37.604162619 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-2lm2s" (UID: "ac0aff21-ec17-47fe-b4a2-b90b5d07786b") : secret "networking-console-plugin-cert" not found
Sep 15 20:58:19.219681 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:19.219067    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pjz7r" event={"ID":"88840482-9724-4d2b-8343-3ed2ad33b258","Type":"ContainerStarted","Data":"d2c29afff6654853de882aa7b916bd1bfb8799bcdb62092d1fc6f97bfd31be4f"}
Sep 15 20:58:19.225224 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:19.225128    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m8fpw" event={"ID":"a70ba96c-ccb0-436a-aa3c-e056cb9a76e3","Type":"ContainerStarted","Data":"3fad7c64570586285f433cd930b46a6e83b656057eaca63af0b6a41e138e9b17"}
Sep 15 20:58:20.232482 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:20.232442    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-dbwkr" event={"ID":"26e0bf54-90c8-4daa-9808-16b7501bc0a3","Type":"ContainerStarted","Data":"83a706bb61a77ea744615369627e5e209cb17666f81eb7315bc01275f4e00531"}
Sep 15 20:58:20.247429 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:20.247369    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-dbwkr" podStartSLOduration=34.32232042 podStartE2EDuration="36.247347158s" totalImagesPullingTime="1.925026738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:57:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:20.246634323 +0000 UTC m=+36.788796977" watchObservedRunningTime="2026-09-15 20:58:20.247347158 +0000 UTC m=+36.789509796"
Sep 15 20:58:20.315795 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:20.315757    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:20.374173 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:20.374138    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:20.577980 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:20.577904    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-574cs\" (UID: \"1d7ebda7-74f1-4762-b7a9-78008fa7f9cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-574cs"
Sep 15 20:58:20.577980 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:20.577945    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:20.577980 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:20.577973    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/04865999-e56e-4a65-9c4a-b52647e7b1db-metrics-tls\") pod \"dns-default-rqmrf\" (UID: \"04865999-e56e-4a65-9c4a-b52647e7b1db\") " pod="openshift-dns/dns-default-rqmrf"
Sep 15 20:58:20.578258 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:20.578087    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:58:20.578258 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:20.578140    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04865999-e56e-4a65-9c4a-b52647e7b1db-metrics-tls podName:04865999-e56e-4a65-9c4a-b52647e7b1db nodeName:}" failed. No retries permitted until 2026-09-15 20:58:24.57812511 +0000 UTC m=+41.120287736 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/04865999-e56e-4a65-9c4a-b52647e7b1db-metrics-tls") pod "dns-default-rqmrf" (UID: "04865999-e56e-4a65-9c4a-b52647e7b1db") : secret "dns-default-metrics-tls" not found
Sep 15 20:58:20.578543 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:20.578522    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 15 20:58:20.578632 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:20.578585    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-samples-operator-tls podName:1d7ebda7-74f1-4762-b7a9-78008fa7f9cc nodeName:}" failed. No retries permitted until 2026-09-15 20:58:24.578568495 +0000 UTC m=+41.120731121 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-574cs" (UID: "1d7ebda7-74f1-4762-b7a9-78008fa7f9cc") : secret "samples-operator-tls" not found
Sep 15 20:58:20.578632 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:20.578611    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 20:58:20.578632 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:20.578628    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-846b7dcc97-fpgwt: secret "image-registry-tls" not found
Sep 15 20:58:20.578775 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:20.578676    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls podName:2243bd69-2640-4dc6-beaa-7e22b414db60 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:24.578661889 +0000 UTC m=+41.120824514 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls") pod "image-registry-846b7dcc97-fpgwt" (UID: "2243bd69-2640-4dc6-beaa-7e22b414db60") : secret "image-registry-tls" not found
Sep 15 20:58:21.081783 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:21.081713    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2lm2s\" (UID: \"ac0aff21-ec17-47fe-b4a2-b90b5d07786b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2lm2s"
Sep 15 20:58:21.081990 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:21.081971    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 15 20:58:21.082117 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:21.082043    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-networking-console-plugin-cert podName:ac0aff21-ec17-47fe-b4a2-b90b5d07786b nodeName:}" failed. No retries permitted until 2026-09-15 20:58:25.082024139 +0000 UTC m=+41.624186755 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-2lm2s" (UID: "ac0aff21-ec17-47fe-b4a2-b90b5d07786b") : secret "networking-console-plugin-cert" not found
Sep 15 20:58:21.896608 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:21.896519    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:21.996830 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:21.996728    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:22.431303 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:22.431273    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:22.482420 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:22.482299    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:24.335909 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:24.335874    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:24.389761 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:24.389722    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:24.615593 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:24.615544    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-574cs\" (UID: \"1d7ebda7-74f1-4762-b7a9-78008fa7f9cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-574cs"
Sep 15 20:58:24.615593 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:24.615594    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:24.615854 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:24.615665    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/04865999-e56e-4a65-9c4a-b52647e7b1db-metrics-tls\") pod \"dns-default-rqmrf\" (UID: \"04865999-e56e-4a65-9c4a-b52647e7b1db\") " pod="openshift-dns/dns-default-rqmrf"
Sep 15 20:58:24.615854 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:24.615735    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:58:24.615854 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:24.615779    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 20:58:24.615854 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:24.615793    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04865999-e56e-4a65-9c4a-b52647e7b1db-metrics-tls podName:04865999-e56e-4a65-9c4a-b52647e7b1db nodeName:}" failed. No retries permitted until 2026-09-15 20:58:32.615776735 +0000 UTC m=+49.157939371 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/04865999-e56e-4a65-9c4a-b52647e7b1db-metrics-tls") pod "dns-default-rqmrf" (UID: "04865999-e56e-4a65-9c4a-b52647e7b1db") : secret "dns-default-metrics-tls" not found
Sep 15 20:58:24.615854 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:24.615798    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-846b7dcc97-fpgwt: secret "image-registry-tls" not found
Sep 15 20:58:24.615854 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:24.615849    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls podName:2243bd69-2640-4dc6-beaa-7e22b414db60 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:32.615834065 +0000 UTC m=+49.157996691 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls") pod "image-registry-846b7dcc97-fpgwt" (UID: "2243bd69-2640-4dc6-beaa-7e22b414db60") : secret "image-registry-tls" not found
Sep 15 20:58:24.616098 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:24.615857    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 15 20:58:24.616098 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:24.615931    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-samples-operator-tls podName:1d7ebda7-74f1-4762-b7a9-78008fa7f9cc nodeName:}" failed. No retries permitted until 2026-09-15 20:58:32.615917178 +0000 UTC m=+49.158079791 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-574cs" (UID: "1d7ebda7-74f1-4762-b7a9-78008fa7f9cc") : secret "samples-operator-tls" not found
Sep 15 20:58:25.119440 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:25.119395    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2lm2s\" (UID: \"ac0aff21-ec17-47fe-b4a2-b90b5d07786b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2lm2s"
Sep 15 20:58:25.119638 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:25.119531    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 15 20:58:25.119638 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:25.119589    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-networking-console-plugin-cert podName:ac0aff21-ec17-47fe-b4a2-b90b5d07786b nodeName:}" failed. No retries permitted until 2026-09-15 20:58:33.119575197 +0000 UTC m=+49.661737810 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-2lm2s" (UID: "ac0aff21-ec17-47fe-b4a2-b90b5d07786b") : secret "networking-console-plugin-cert" not found
Sep 15 20:58:25.298847 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:25.298804    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:25.399104 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:25.399007    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:26.236679 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:26.236636    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:29.262701 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:29.262621    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jsn2f" event={"ID":"dcfdcceb-cea6-4f5a-958f-a1b558bc1a1f","Type":"ContainerStarted","Data":"1ee60dcf38da5b9169bf5255c39ef82e4323b3d3259d6aac8659cf86a6232558"}
Sep 15 20:58:29.266692 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:29.266639    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-7t46m" event={"ID":"c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf","Type":"ContainerStarted","Data":"cd766895d373fda862318761e215e1565d23f09148108c7bd0d3b8c0568bacc2"}
Sep 15 20:58:29.275611 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:29.275558    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-7t46m"
Sep 15 20:58:29.276871 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:29.276831    2564 patch_prober.go:28] interesting pod/console-operator-696884fb9c-7t46m container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.9:8443/readyz\": dial tcp 10.132.0.9:8443: connect: connection refused" start-of-body=
Sep 15 20:58:29.276972 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:29.276890    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-696884fb9c-7t46m" podUID="c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.9:8443/readyz\": dial tcp 10.132.0.9:8443: connect: connection refused"
Sep 15 20:58:29.288989 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:29.288430    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-lzdl4" event={"ID":"90967cfe-f101-4cf3-bdc5-b0e156f578c1","Type":"ContainerStarted","Data":"6c0e5d44223cfb9df3b29c23a09fe06c07a1c3147c12624849c0e0595cd69cf8"}
Sep 15 20:58:29.305111 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:29.305007    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-7t46m" podStartSLOduration=32.036588361 podStartE2EDuration="43.304983447s" totalImagesPullingTime="11.268395086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:57:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:29.303379201 +0000 UTC m=+45.845541855" watchObservedRunningTime="2026-09-15 20:58:29.304983447 +0000 UTC m=+45.847146083"
Sep 15 20:58:29.321424 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:29.319132    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-lzdl4" podStartSLOduration=26.7263102 podStartE2EDuration="38.319114599s" totalImagesPullingTime="11.592804399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:29.317812329 +0000 UTC m=+45.859974997" watchObservedRunningTime="2026-09-15 20:58:29.319114599 +0000 UTC m=+45.861277233"
Sep 15 20:58:30.294467 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.294439    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-7t46m_c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf/console-operator/0.log"
Sep 15 20:58:30.294877 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.294482    2564 generic.go:356] "Generic (PLEG): container finished" podID="c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf" containerID="cd766895d373fda862318761e215e1565d23f09148108c7bd0d3b8c0568bacc2" exitCode=255
Sep 15 20:58:30.294877 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.294599    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-7t46m" event={"ID":"c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf","Type":"ContainerDied","Data":"cd766895d373fda862318761e215e1565d23f09148108c7bd0d3b8c0568bacc2"}
Sep 15 20:58:30.294877 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.294847    2564 scope.go:117] "RemoveContainer" containerID="cd766895d373fda862318761e215e1565d23f09148108c7bd0d3b8c0568bacc2"
Sep 15 20:58:30.296377 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.296336    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pjz7r" event={"ID":"88840482-9724-4d2b-8343-3ed2ad33b258","Type":"ContainerStarted","Data":"89e43f8a99aec8759ccf32ab157ff00a736a783435d37651824a4309a874999a"}
Sep 15 20:58:30.299445 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.299421    2564 generic.go:356] "Generic (PLEG): container finished" podID="b24f6335-59f4-4b7a-8548-56465ed60966" containerID="a8d945df5fa1ebac2b74dca24aeaec429ed6970b6aafd61b020d7caebb419016" exitCode=0
Sep 15 20:58:30.299582 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.299528    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8vrp" event={"ID":"b24f6335-59f4-4b7a-8548-56465ed60966","Type":"ContainerDied","Data":"a8d945df5fa1ebac2b74dca24aeaec429ed6970b6aafd61b020d7caebb419016"}
Sep 15 20:58:30.301384 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.301364    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5mvsw" event={"ID":"2aae8637-fdbb-4c3c-a878-ec6ea9a25c7b","Type":"ContainerStarted","Data":"8ce3e3b621433af06ae39c0ab8e0fa72f17b464ca5904bc5b20057438ef69b3f"}
Sep 15 20:58:30.302965 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.302933    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m8fpw" event={"ID":"a70ba96c-ccb0-436a-aa3c-e056cb9a76e3","Type":"ContainerStarted","Data":"f55c0e8ce87204c732f2e12bebeaf6d0ba65cb5fa5b320d07bac3426743533f3"}
Sep 15 20:58:30.303054 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.303011    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:58:30.304554 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.304531    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ppbrv" event={"ID":"f3b99f56-873f-40d8-abf3-b0c02523b2f7","Type":"ContainerStarted","Data":"4633ea3e4451167ea2ab658f86e7085ee63ac9c2de4b96a350acbb03d81044d2"}
Sep 15 20:58:30.327947 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.327161    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-m8fpw" podStartSLOduration=35.442994668 podStartE2EDuration="46.327137991s" totalImagesPullingTime="10.884143323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:57:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:30.326550753 +0000 UTC m=+46.868713412" watchObservedRunningTime="2026-09-15 20:58:30.327137991 +0000 UTC m=+46.869300626"
Sep 15 20:58:30.344173 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.344097    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pjz7r" podStartSLOduration=35.455901966 podStartE2EDuration="46.344073991s" totalImagesPullingTime="10.888172025s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:57:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:30.342681638 +0000 UTC m=+46.884844276" watchObservedRunningTime="2026-09-15 20:58:30.344073991 +0000 UTC m=+46.886236630"
Sep 15 20:58:30.381952 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.381900    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jsn2f" podStartSLOduration=28.732305009 podStartE2EDuration="40.381856552s" totalImagesPullingTime="11.649551543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:57:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:30.381045299 +0000 UTC m=+46.923207955" watchObservedRunningTime="2026-09-15 20:58:30.381856552 +0000 UTC m=+46.924019223"
Sep 15 20:58:30.400610 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.400559    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ppbrv" podStartSLOduration=28.805961445 podStartE2EDuration="40.400541933s" totalImagesPullingTime="11.594580488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:57:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:30.399331989 +0000 UTC m=+46.941494650" watchObservedRunningTime="2026-09-15 20:58:30.400541933 +0000 UTC m=+46.942704641"
Sep 15 20:58:30.417708 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.417650    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5mvsw" podStartSLOduration=2.85243094 podStartE2EDuration="14.417626667s" totalImagesPullingTime="11.565195727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:58:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:30.416057944 +0000 UTC m=+46.958220606" watchObservedRunningTime="2026-09-15 20:58:30.417626667 +0000 UTC m=+46.959789303"
Sep 15 20:58:30.912085 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.912039    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:30.930485 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.930445    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-snq76"]
Sep 15 20:58:30.955024 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.954991    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-snq76"]
Sep 15 20:58:30.955158 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.955102    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-snq76"
Sep 15 20:58:30.958245 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.958195    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 15 20:58:30.958245 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.958195    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 15 20:58:30.958427 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.958303    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-tx2pb\""
Sep 15 20:58:30.977258 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.977229    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2sfj\" (UniqueName: \"kubernetes.io/projected/d6e9962c-32fc-4fd1-8baa-aed7a396f606-kube-api-access-f2sfj\") pod \"migrator-8b4d697c5-snq76\" (UID: \"d6e9962c-32fc-4fd1-8baa-aed7a396f606\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-snq76"
Sep 15 20:58:30.978283 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:30.978259    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:31.039510 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:31.036915    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:31.078016 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:31.077973    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2sfj\" (UniqueName: \"kubernetes.io/projected/d6e9962c-32fc-4fd1-8baa-aed7a396f606-kube-api-access-f2sfj\") pod \"migrator-8b4d697c5-snq76\" (UID: \"d6e9962c-32fc-4fd1-8baa-aed7a396f606\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-snq76"
Sep 15 20:58:31.087499 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:31.087469    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2sfj\" (UniqueName: \"kubernetes.io/projected/d6e9962c-32fc-4fd1-8baa-aed7a396f606-kube-api-access-f2sfj\") pod \"migrator-8b4d697c5-snq76\" (UID: \"d6e9962c-32fc-4fd1-8baa-aed7a396f606\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-snq76"
Sep 15 20:58:31.138341 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:31.138305    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:31.191976 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:31.191886    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:31.265259 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:31.265224    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-snq76"
Sep 15 20:58:31.309922 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:31.309899    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-7t46m_c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf/console-operator/1.log"
Sep 15 20:58:31.310369 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:31.310347    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-7t46m_c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf/console-operator/0.log"
Sep 15 20:58:31.310482 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:31.310380    2564 generic.go:356] "Generic (PLEG): container finished" podID="c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf" containerID="017b5f65c98673dfd65116421db81266ddf24674ed136167ae1275eac8b613c0" exitCode=255
Sep 15 20:58:31.310482 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:31.310443    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-7t46m" event={"ID":"c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf","Type":"ContainerDied","Data":"017b5f65c98673dfd65116421db81266ddf24674ed136167ae1275eac8b613c0"}
Sep 15 20:58:31.310630 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:31.310567    2564 scope.go:117] "RemoveContainer" containerID="cd766895d373fda862318761e215e1565d23f09148108c7bd0d3b8c0568bacc2"
Sep 15 20:58:31.311038 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:31.311022    2564 scope.go:117] "RemoveContainer" containerID="017b5f65c98673dfd65116421db81266ddf24674ed136167ae1275eac8b613c0"
Sep 15 20:58:31.311325 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:31.311222    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-7t46m_openshift-console-operator(c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf)\"" pod="openshift-console-operator/console-operator-696884fb9c-7t46m" podUID="c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf"
Sep 15 20:58:31.314262 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:31.314240    2564 generic.go:356] "Generic (PLEG): container finished" podID="b24f6335-59f4-4b7a-8548-56465ed60966" containerID="8368193e9cd9878bb7b7272b6762e18ec30f3f26abb6b0155ca3a8217095152e" exitCode=0
Sep 15 20:58:31.314366 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:31.314305    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8vrp" event={"ID":"b24f6335-59f4-4b7a-8548-56465ed60966","Type":"ContainerDied","Data":"8368193e9cd9878bb7b7272b6762e18ec30f3f26abb6b0155ca3a8217095152e"}
Sep 15 20:58:31.378442 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:31.378409    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:31.460153 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:31.460119    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:31.564745 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:31.564710    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-snq76"]
Sep 15 20:58:31.567252 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:58:31.567224    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6e9962c_32fc_4fd1_8baa_aed7a396f606.slice/crio-62a7a3c9bf7985afeaaceca64818e810f4f2d42cb2236e03d5117327989d26b2 WatchSource:0}: Error finding container 62a7a3c9bf7985afeaaceca64818e810f4f2d42cb2236e03d5117327989d26b2: Status 404 returned error can't find the container with id 62a7a3c9bf7985afeaaceca64818e810f4f2d42cb2236e03d5117327989d26b2
Sep 15 20:58:31.834789 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:31.834708    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:31.939345 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:31.939307    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:32.319470 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:32.319431    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-snq76" event={"ID":"d6e9962c-32fc-4fd1-8baa-aed7a396f606","Type":"ContainerStarted","Data":"62a7a3c9bf7985afeaaceca64818e810f4f2d42cb2236e03d5117327989d26b2"}
Sep 15 20:58:32.322405 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:32.321287    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-7t46m_c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf/console-operator/1.log"
Sep 15 20:58:32.322405 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:32.321733    2564 scope.go:117] "RemoveContainer" containerID="017b5f65c98673dfd65116421db81266ddf24674ed136167ae1275eac8b613c0"
Sep 15 20:58:32.322405 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:32.321910    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-7t46m_openshift-console-operator(c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf)\"" pod="openshift-console-operator/console-operator-696884fb9c-7t46m" podUID="c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf"
Sep 15 20:58:32.326643 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:32.326616    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8vrp" event={"ID":"b24f6335-59f4-4b7a-8548-56465ed60966","Type":"ContainerStarted","Data":"7296d9e2bb5c7878e02aa6f848d0974781acd91b62013678e9f15c9e15c0c2b0"}
Sep 15 20:58:32.360778 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:32.360724    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-b8vrp" podStartSLOduration=8.991090963 podStartE2EDuration="48.360706778s" totalImagesPullingTime="39.369615815s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:57:44 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:32.358968648 +0000 UTC m=+48.901131314" watchObservedRunningTime="2026-09-15 20:58:32.360706778 +0000 UTC m=+48.902869413"
Sep 15 20:58:32.480636 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:32.480597    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:32.580922 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:32.580832    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:32.690450 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:32.690416    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-574cs\" (UID: \"1d7ebda7-74f1-4762-b7a9-78008fa7f9cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-574cs"
Sep 15 20:58:32.690637 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:32.690454    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:32.690637 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:32.690482    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/04865999-e56e-4a65-9c4a-b52647e7b1db-metrics-tls\") pod \"dns-default-rqmrf\" (UID: \"04865999-e56e-4a65-9c4a-b52647e7b1db\") " pod="openshift-dns/dns-default-rqmrf"
Sep 15 20:58:32.690637 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:32.690599    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 20:58:32.690637 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:32.690606    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 15 20:58:32.690791 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:32.690648    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04865999-e56e-4a65-9c4a-b52647e7b1db-metrics-tls podName:04865999-e56e-4a65-9c4a-b52647e7b1db nodeName:}" failed. No retries permitted until 2026-09-15 20:58:48.690634557 +0000 UTC m=+65.232797170 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/04865999-e56e-4a65-9c4a-b52647e7b1db-metrics-tls") pod "dns-default-rqmrf" (UID: "04865999-e56e-4a65-9c4a-b52647e7b1db") : secret "dns-default-metrics-tls" not found
Sep 15 20:58:32.690791 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:32.690693    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-samples-operator-tls podName:1d7ebda7-74f1-4762-b7a9-78008fa7f9cc nodeName:}" failed. No retries permitted until 2026-09-15 20:58:48.690678219 +0000 UTC m=+65.232840836 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-574cs" (UID: "1d7ebda7-74f1-4762-b7a9-78008fa7f9cc") : secret "samples-operator-tls" not found
Sep 15 20:58:32.690791 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:32.690608    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 20:58:32.690791 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:32.690710    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-846b7dcc97-fpgwt: secret "image-registry-tls" not found
Sep 15 20:58:32.690791 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:32.690751    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls podName:2243bd69-2640-4dc6-beaa-7e22b414db60 nodeName:}" failed. No retries permitted until 2026-09-15 20:58:48.690739457 +0000 UTC m=+65.232902092 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls") pod "image-registry-846b7dcc97-fpgwt" (UID: "2243bd69-2640-4dc6-beaa-7e22b414db60") : secret "image-registry-tls" not found
Sep 15 20:58:33.195378 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:33.195342    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2lm2s\" (UID: \"ac0aff21-ec17-47fe-b4a2-b90b5d07786b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2lm2s"
Sep 15 20:58:33.195582 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:33.195535    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 15 20:58:33.195626 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:33.195611    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-networking-console-plugin-cert podName:ac0aff21-ec17-47fe-b4a2-b90b5d07786b nodeName:}" failed. No retries permitted until 2026-09-15 20:58:49.195594702 +0000 UTC m=+65.737757329 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-2lm2s" (UID: "ac0aff21-ec17-47fe-b4a2-b90b5d07786b") : secret "networking-console-plugin-cert" not found
Sep 15 20:58:33.435230 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:33.435187    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-xtqhj"]
Sep 15 20:58:33.449817 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:33.449740    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-xtqhj"]
Sep 15 20:58:33.449954 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:33.449870    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-xtqhj"
Sep 15 20:58:33.453885 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:33.453865    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 15 20:58:33.454033 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:33.453864    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 15 20:58:33.454033 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:33.453865    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 15 20:58:33.455213 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:33.455197    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 15 20:58:33.455299 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:33.455216    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-z4n2h\""
Sep 15 20:58:33.497597 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:33.497561    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3de9bc9f-bf68-4016-b564-47d820e1c0dd-signing-key\") pod \"service-ca-568869fdc5-xtqhj\" (UID: \"3de9bc9f-bf68-4016-b564-47d820e1c0dd\") " pod="openshift-service-ca/service-ca-568869fdc5-xtqhj"
Sep 15 20:58:33.497756 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:33.497607    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3de9bc9f-bf68-4016-b564-47d820e1c0dd-signing-cabundle\") pod \"service-ca-568869fdc5-xtqhj\" (UID: \"3de9bc9f-bf68-4016-b564-47d820e1c0dd\") " pod="openshift-service-ca/service-ca-568869fdc5-xtqhj"
Sep 15 20:58:33.497756 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:33.497636    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkrll\" (UniqueName: \"kubernetes.io/projected/3de9bc9f-bf68-4016-b564-47d820e1c0dd-kube-api-access-pkrll\") pod \"service-ca-568869fdc5-xtqhj\" (UID: \"3de9bc9f-bf68-4016-b564-47d820e1c0dd\") " pod="openshift-service-ca/service-ca-568869fdc5-xtqhj"
Sep 15 20:58:33.536509 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:33.536454    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:33.590630 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:33.590594    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:33.598589 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:33.598560    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3de9bc9f-bf68-4016-b564-47d820e1c0dd-signing-key\") pod \"service-ca-568869fdc5-xtqhj\" (UID: \"3de9bc9f-bf68-4016-b564-47d820e1c0dd\") " pod="openshift-service-ca/service-ca-568869fdc5-xtqhj"
Sep 15 20:58:33.598728 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:33.598602    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3de9bc9f-bf68-4016-b564-47d820e1c0dd-signing-cabundle\") pod \"service-ca-568869fdc5-xtqhj\" (UID: \"3de9bc9f-bf68-4016-b564-47d820e1c0dd\") " pod="openshift-service-ca/service-ca-568869fdc5-xtqhj"
Sep 15 20:58:33.598728 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:33.598622    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkrll\" (UniqueName: \"kubernetes.io/projected/3de9bc9f-bf68-4016-b564-47d820e1c0dd-kube-api-access-pkrll\") pod \"service-ca-568869fdc5-xtqhj\" (UID: \"3de9bc9f-bf68-4016-b564-47d820e1c0dd\") " pod="openshift-service-ca/service-ca-568869fdc5-xtqhj"
Sep 15 20:58:33.599185 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:33.599154    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3de9bc9f-bf68-4016-b564-47d820e1c0dd-signing-cabundle\") pod \"service-ca-568869fdc5-xtqhj\" (UID: \"3de9bc9f-bf68-4016-b564-47d820e1c0dd\") " pod="openshift-service-ca/service-ca-568869fdc5-xtqhj"
Sep 15 20:58:33.601361 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:33.601336    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3de9bc9f-bf68-4016-b564-47d820e1c0dd-signing-key\") pod \"service-ca-568869fdc5-xtqhj\" (UID: \"3de9bc9f-bf68-4016-b564-47d820e1c0dd\") " pod="openshift-service-ca/service-ca-568869fdc5-xtqhj"
Sep 15 20:58:33.606953 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:33.606928    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkrll\" (UniqueName: \"kubernetes.io/projected/3de9bc9f-bf68-4016-b564-47d820e1c0dd-kube-api-access-pkrll\") pod \"service-ca-568869fdc5-xtqhj\" (UID: \"3de9bc9f-bf68-4016-b564-47d820e1c0dd\") " pod="openshift-service-ca/service-ca-568869fdc5-xtqhj"
Sep 15 20:58:33.762964 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:33.762939    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-xtqhj"
Sep 15 20:58:34.059725 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:34.059630    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-xtqhj"]
Sep 15 20:58:34.063645 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:58:34.063619    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3de9bc9f_bf68_4016_b564_47d820e1c0dd.slice/crio-6fef985b239b3509bf45fc25b8277ba98daba675874ffa81996fb33448f2208f WatchSource:0}: Error finding container 6fef985b239b3509bf45fc25b8277ba98daba675874ffa81996fb33448f2208f: Status 404 returned error can't find the container with id 6fef985b239b3509bf45fc25b8277ba98daba675874ffa81996fb33448f2208f
Sep 15 20:58:34.338363 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:34.338270    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-xtqhj" event={"ID":"3de9bc9f-bf68-4016-b564-47d820e1c0dd","Type":"ContainerStarted","Data":"0113a80d4ad45d737d371cda59ed43986d248e962692eb341927b5331105ae5d"}
Sep 15 20:58:34.338363 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:34.338328    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-xtqhj" event={"ID":"3de9bc9f-bf68-4016-b564-47d820e1c0dd","Type":"ContainerStarted","Data":"6fef985b239b3509bf45fc25b8277ba98daba675874ffa81996fb33448f2208f"}
Sep 15 20:58:34.340148 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:34.340115    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-snq76" event={"ID":"d6e9962c-32fc-4fd1-8baa-aed7a396f606","Type":"ContainerStarted","Data":"e56e3a1f3bf7a6f37ee8b06083a5d01491c7c2556db02c9ed8166744c079d126"}
Sep 15 20:58:34.340265 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:34.340164    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-snq76" event={"ID":"d6e9962c-32fc-4fd1-8baa-aed7a396f606","Type":"ContainerStarted","Data":"ac60871ecae86829c3894c61acee44c49baf2b49c48639ed9f7faecbf0e0361c"}
Sep 15 20:58:34.354734 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:34.354675    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-xtqhj" podStartSLOduration=1.354656107 podStartE2EDuration="1.354656107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:58:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:34.354617007 +0000 UTC m=+50.896779644" watchObservedRunningTime="2026-09-15 20:58:34.354656107 +0000 UTC m=+50.896818743"
Sep 15 20:58:34.369839 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:34.369795    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-snq76" podStartSLOduration=2.260091059 podStartE2EDuration="4.369779434s" totalImagesPullingTime="2.109688375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:58:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:34.369037573 +0000 UTC m=+50.911200220" watchObservedRunningTime="2026-09-15 20:58:34.369779434 +0000 UTC m=+50.911942065"
Sep 15 20:58:34.418546 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:34.418474    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:35.238313 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:35.238262    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:35.343992 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:35.343962    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:35.485467 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:35.485380    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:35.585799 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:35.585678    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:35.752517 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:35.752470    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:35.889943 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:35.889902    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:36.696142 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:36.696102    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:37.132950 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:37.132902    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-7t46m"
Sep 15 20:58:37.133304 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:37.133291    2564 scope.go:117] "RemoveContainer" containerID="017b5f65c98673dfd65116421db81266ddf24674ed136167ae1275eac8b613c0"
Sep 15 20:58:37.133510 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:37.133469    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-7t46m_openshift-console-operator(c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf)\"" pod="openshift-console-operator/console-operator-696884fb9c-7t46m" podUID="c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf"
Sep 15 20:58:37.314566 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:37.314525    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:37.416678 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:37.416589    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:37.518231 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:37.518198    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:37.701213 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:37.701122    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:38.092759 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:38.092670    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:38.489644 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:38.489603    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:38.882185 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:38.882137    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:38.945016 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:38.944990    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:39.089617 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:39.089578    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:39.275286 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:39.275200    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-7t46m"
Sep 15 20:58:39.275611 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:39.275597    2564 scope.go:117] "RemoveContainer" containerID="017b5f65c98673dfd65116421db81266ddf24674ed136167ae1275eac8b613c0"
Sep 15 20:58:39.275780 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:58:39.275762    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-7t46m_openshift-console-operator(c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf)\"" pod="openshift-console-operator/console-operator-696884fb9c-7t46m" podUID="c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf"
Sep 15 20:58:39.494055 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:39.494018    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:39.545465 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:39.545380    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:39.892551 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:39.892506    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:40.086089 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:40.086040    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:40.691967 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:40.691929    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:40.792505 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:40.792456    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:40.990740 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:40.990651    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:41.296295 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:41.296215    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:41.491574 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:41.491535    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:41.670900 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:41.670838    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:41.771344 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:41.771307    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:42.091849 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:42.091760    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:42.699021 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:42.698987    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:43.290507 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:43.290458    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:43.489124 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:43.489081    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:43.691766 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:43.691727    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:44.087245 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:44.087158    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:44.887859 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:44.887818    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:45.092064 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:45.092024    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:45.291584 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:45.291484    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:45.490441 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:45.490402    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:48.724242 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:48.724205    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-574cs\" (UID: \"1d7ebda7-74f1-4762-b7a9-78008fa7f9cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-574cs"
Sep 15 20:58:48.724242 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:48.724245    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:48.724723 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:48.724270    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/04865999-e56e-4a65-9c4a-b52647e7b1db-metrics-tls\") pod \"dns-default-rqmrf\" (UID: \"04865999-e56e-4a65-9c4a-b52647e7b1db\") " pod="openshift-dns/dns-default-rqmrf"
Sep 15 20:58:48.726956 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:48.726923    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls\") pod \"image-registry-846b7dcc97-fpgwt\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") " pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:48.727079 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:48.726923    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/04865999-e56e-4a65-9c4a-b52647e7b1db-metrics-tls\") pod \"dns-default-rqmrf\" (UID: \"04865999-e56e-4a65-9c4a-b52647e7b1db\") " pod="openshift-dns/dns-default-rqmrf"
Sep 15 20:58:48.727079 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:48.726966    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1d7ebda7-74f1-4762-b7a9-78008fa7f9cc-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-574cs\" (UID: \"1d7ebda7-74f1-4762-b7a9-78008fa7f9cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-574cs"
Sep 15 20:58:48.923426 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:48.923391    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-fh6zd\""
Sep 15 20:58:48.931596 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:48.931572    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-574cs"
Sep 15 20:58:48.950966 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:48.950941    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xpshw\""
Sep 15 20:58:48.959123 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:48.959097    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:48.972632 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:48.972596    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p626r\""
Sep 15 20:58:48.980764 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:48.980738    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rqmrf"
Sep 15 20:58:49.204307 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:49.204281    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-574cs"]
Sep 15 20:58:49.227871 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:49.227685    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2lm2s\" (UID: \"ac0aff21-ec17-47fe-b4a2-b90b5d07786b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2lm2s"
Sep 15 20:58:49.230345 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:49.230317    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac0aff21-ec17-47fe-b4a2-b90b5d07786b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-2lm2s\" (UID: \"ac0aff21-ec17-47fe-b4a2-b90b5d07786b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2lm2s"
Sep 15 20:58:49.246449 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:49.246418    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-846b7dcc97-fpgwt"]
Sep 15 20:58:49.248760 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:58:49.248685    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2243bd69_2640_4dc6_beaa_7e22b414db60.slice/crio-075c7e8868b48fd5c3cbfce2506001c379ff691074bd4ecb3618c59766f53445 WatchSource:0}: Error finding container 075c7e8868b48fd5c3cbfce2506001c379ff691074bd4ecb3618c59766f53445: Status 404 returned error can't find the container with id 075c7e8868b48fd5c3cbfce2506001c379ff691074bd4ecb3618c59766f53445
Sep 15 20:58:49.261017 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:49.260991    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rqmrf"]
Sep 15 20:58:49.264151 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:58:49.264118    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04865999_e56e_4a65_9c4a_b52647e7b1db.slice/crio-5f3150bb9169551ba47fbae1c345013e431c554c1ccccde0948396fb388c9e67 WatchSource:0}: Error finding container 5f3150bb9169551ba47fbae1c345013e431c554c1ccccde0948396fb388c9e67: Status 404 returned error can't find the container with id 5f3150bb9169551ba47fbae1c345013e431c554c1ccccde0948396fb388c9e67
Sep 15 20:58:49.393655 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:49.393621    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt" event={"ID":"2243bd69-2640-4dc6-beaa-7e22b414db60","Type":"ContainerStarted","Data":"3de31086882d5b0c696c11f710e50cf3fc6b9a5e6aaa686927b28715a5b76703"}
Sep 15 20:58:49.393856 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:49.393671    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt" event={"ID":"2243bd69-2640-4dc6-beaa-7e22b414db60","Type":"ContainerStarted","Data":"075c7e8868b48fd5c3cbfce2506001c379ff691074bd4ecb3618c59766f53445"}
Sep 15 20:58:49.393856 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:49.393736    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:58:49.394990 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:49.394964    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rqmrf" event={"ID":"04865999-e56e-4a65-9c4a-b52647e7b1db","Type":"ContainerStarted","Data":"5f3150bb9169551ba47fbae1c345013e431c554c1ccccde0948396fb388c9e67"}
Sep 15 20:58:49.396093 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:49.396073    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-574cs" event={"ID":"1d7ebda7-74f1-4762-b7a9-78008fa7f9cc","Type":"ContainerStarted","Data":"6a57959204dbf19064f49239ec9f20ac7c71c217e3adc3eabffef7e1cee4394c"}
Sep 15 20:58:49.415831 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:49.415770    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt" podStartSLOduration=58.415750468 podStartE2EDuration="58.415750468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:57:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:49.411406403 +0000 UTC m=+65.953569053" watchObservedRunningTime="2026-09-15 20:58:49.415750468 +0000 UTC m=+65.957913104"
Sep 15 20:58:49.439797 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:49.439764    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-wlh8s\""
Sep 15 20:58:49.448127 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:49.448103    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2lm2s"
Sep 15 20:58:49.714697 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:49.714664    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-2lm2s"]
Sep 15 20:58:49.718216 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:58:49.718188    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac0aff21_ec17_47fe_b4a2_b90b5d07786b.slice/crio-6bf4048910db88c3966869bb2adf93a0b10e53faa90a080901c480d3cb037896 WatchSource:0}: Error finding container 6bf4048910db88c3966869bb2adf93a0b10e53faa90a080901c480d3cb037896: Status 404 returned error can't find the container with id 6bf4048910db88c3966869bb2adf93a0b10e53faa90a080901c480d3cb037896
Sep 15 20:58:49.731837 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:49.731805    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs\") pod \"network-metrics-daemon-4dws2\" (UID: \"ce801675-8966-401c-b202-6631f49bdea0\") " pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:58:49.734591 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:49.734558    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce801675-8966-401c-b202-6631f49bdea0-metrics-certs\") pod \"network-metrics-daemon-4dws2\" (UID: \"ce801675-8966-401c-b202-6631f49bdea0\") " pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:58:49.994784 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:49.994746    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-r6tvw\""
Sep 15 20:58:50.002328 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:50.002294    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4dws2"
Sep 15 20:58:50.280503 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:50.280354    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4dws2"]
Sep 15 20:58:50.285144 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:58:50.285108    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce801675_8966_401c_b202_6631f49bdea0.slice/crio-3288078a2843b310b831fde3e08632d84b61fde64d8ca197c9b73a65f435fb77 WatchSource:0}: Error finding container 3288078a2843b310b831fde3e08632d84b61fde64d8ca197c9b73a65f435fb77: Status 404 returned error can't find the container with id 3288078a2843b310b831fde3e08632d84b61fde64d8ca197c9b73a65f435fb77
Sep 15 20:58:50.403225 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:50.403187    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2lm2s" event={"ID":"ac0aff21-ec17-47fe-b4a2-b90b5d07786b","Type":"ContainerStarted","Data":"6bf4048910db88c3966869bb2adf93a0b10e53faa90a080901c480d3cb037896"}
Sep 15 20:58:50.405249 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:50.404752    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4dws2" event={"ID":"ce801675-8966-401c-b202-6631f49bdea0","Type":"ContainerStarted","Data":"3288078a2843b310b831fde3e08632d84b61fde64d8ca197c9b73a65f435fb77"}
Sep 15 20:58:51.894512 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:51.894405    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:51.945119 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:51.945080    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:51.966609 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:51.966581    2564 scope.go:117] "RemoveContainer" containerID="017b5f65c98673dfd65116421db81266ddf24674ed136167ae1275eac8b613c0"
Sep 15 20:58:52.415883 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:52.415818    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-574cs" event={"ID":"1d7ebda7-74f1-4762-b7a9-78008fa7f9cc","Type":"ContainerStarted","Data":"393f7cc8efe2b565552121b8522b7f769c5ad737baacc7dc00b29c62efa33b2b"}
Sep 15 20:58:52.420480 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:52.420456    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rqmrf" event={"ID":"04865999-e56e-4a65-9c4a-b52647e7b1db","Type":"ContainerStarted","Data":"897e548616291f29a758178bf0b361718d8484ea5e49e2e46bd550ff4723af64"}
Sep 15 20:58:52.423610 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:52.423521    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2lm2s" event={"ID":"ac0aff21-ec17-47fe-b4a2-b90b5d07786b","Type":"ContainerStarted","Data":"19fc440d898f66bf593813d00d1b42d43f90b77d0c2d04f70e697ace39ef501d"}
Sep 15 20:58:52.428524 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:52.428396    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-7t46m_c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf/console-operator/1.log"
Sep 15 20:58:52.428524 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:52.428442    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-7t46m" event={"ID":"c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf","Type":"ContainerStarted","Data":"29d82595283bf94f717cc948e88f978d40d0c12bf4b0263b216fb9c2bf41d7ee"}
Sep 15 20:58:52.432608 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:52.431636    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-7t46m"
Sep 15 20:58:52.433135 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:52.432773    2564 patch_prober.go:28] interesting pod/console-operator-696884fb9c-7t46m container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.9:8443/readyz\": dial tcp 10.132.0.9:8443: connect: connection refused" start-of-body=
Sep 15 20:58:52.433135 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:52.432823    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-696884fb9c-7t46m" podUID="c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.9:8443/readyz\": dial tcp 10.132.0.9:8443: connect: connection refused"
Sep 15 20:58:52.471726 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:52.471665    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-2lm2s" podStartSLOduration=32.888076234 podStartE2EDuration="35.471621781s" totalImagesPullingTime="2.583545547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:58:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:52.447647636 +0000 UTC m=+68.989810273" watchObservedRunningTime="2026-09-15 20:58:52.471621781 +0000 UTC m=+69.013784417"
Sep 15 20:58:53.180711 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:53.180678    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:53.280776 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:53.280733    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:53.434567 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:53.434473    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rqmrf" event={"ID":"04865999-e56e-4a65-9c4a-b52647e7b1db","Type":"ContainerStarted","Data":"65e4e2ac563deb4c0385e1d07059430005c0d7194d901ae118fdddb0d4d13e52"}
Sep 15 20:58:53.434729 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:53.434654    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rqmrf"
Sep 15 20:58:53.436436 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:53.436414    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-574cs" event={"ID":"1d7ebda7-74f1-4762-b7a9-78008fa7f9cc","Type":"ContainerStarted","Data":"10246538713b2bfde67cf132def59c01a734350f2603a93417f2e00d0f20890f"}
Sep 15 20:58:53.438278 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:53.438254    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4dws2" event={"ID":"ce801675-8966-401c-b202-6631f49bdea0","Type":"ContainerStarted","Data":"4ed0ede978cf33df4386494bd746d0f58a6f5e595ab5ae4e675a933fb2d662cb"}
Sep 15 20:58:53.438388 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:53.438300    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4dws2" event={"ID":"ce801675-8966-401c-b202-6631f49bdea0","Type":"ContainerStarted","Data":"8b8f65478bfd1b5b79382774e9ec4344261ff8645d1463543b376eb26054dba3"}
Sep 15 20:58:53.442473 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:53.442455    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-7t46m"
Sep 15 20:58:53.452808 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:53.452774    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rqmrf" podStartSLOduration=34.414970879 podStartE2EDuration="37.452762234s" totalImagesPullingTime="3.037791355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:58:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:53.451592867 +0000 UTC m=+69.993755546" watchObservedRunningTime="2026-09-15 20:58:53.452762234 +0000 UTC m=+69.994924869"
Sep 15 20:58:53.467323 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:53.467277    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-574cs" podStartSLOduration=66.403896924 podStartE2EDuration="1m9.467261643s" totalImagesPullingTime="3.063364719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:57:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:53.466982455 +0000 UTC m=+70.009145109" watchObservedRunningTime="2026-09-15 20:58:53.467261643 +0000 UTC m=+70.009424283"
Sep 15 20:58:53.471115 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:53.471088    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:53.502905 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:53.502842    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4dws2" podStartSLOduration=67.485547693 podStartE2EDuration="1m9.502819221s" totalImagesPullingTime="2.017271528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:57:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:53.501928078 +0000 UTC m=+70.044090731" watchObservedRunningTime="2026-09-15 20:58:53.502819221 +0000 UTC m=+70.044981856"
Sep 15 20:58:53.656081 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:53.656039    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:53.719057 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:53.718962    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:54.048626 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:54.048541    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:54.106199 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:54.106157    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:54.251530 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:54.251481    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:54.299279 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:54.299202    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:55.070349 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:55.070314    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:55.260195 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:55.260161    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:55.711074 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:55.711030    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:56.281049 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:56.281018    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:58.049112 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.049084    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-rlf4s"]
Sep 15 20:58:58.079235 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.079207    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-qlv8v"]
Sep 15 20:58:58.079448 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.079425    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-rlf4s"
Sep 15 20:58:58.082616 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.082594    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 15 20:58:58.082928 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.082900    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 15 20:58:58.083204 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.083182    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 15 20:58:58.083530 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.083513    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 15 20:58:58.083864 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.083848    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-r56v7\""
Sep 15 20:58:58.090547 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.090526    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 15 20:58:58.096838 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.096817    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d5cd9679-h825p"]
Sep 15 20:58:58.096956 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.096941    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qlv8v"
Sep 15 20:58:58.100470 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.100141    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 15 20:58:58.100470 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.100174    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 15 20:58:58.100470 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.100340    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 15 20:58:58.101200 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.101179    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dckld\""
Sep 15 20:58:58.104535 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.104514    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 15 20:58:58.118882 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.118848    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-ddcc97c8b-72s7f"]
Sep 15 20:58:58.119024 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.118914    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d5cd9679-h825p"
Sep 15 20:58:58.121466 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.121435    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 15 20:58:58.121466 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.121457    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 15 20:58:58.122361 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.122341    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 15 20:58:58.122480 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.122379    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 15 20:58:58.128666 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.128619    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:58.141108 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.141078    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9"]
Sep 15 20:58:58.141243 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.141188    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-ddcc97c8b-72s7f"
Sep 15 20:58:58.144016 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.143960    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 15 20:58:58.144016 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.143936    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 15 20:58:58.144016 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.143995    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 15 20:58:58.144312 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.144296    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 15 20:58:58.144312 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.144307    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 15 20:58:58.144441 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.144313    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tlf24\""
Sep 15 20:58:58.144441 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.144357    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 15 20:58:58.161116 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.161067    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-qlv8v"]
Sep 15 20:58:58.161116 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.161118    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-846b7dcc97-fpgwt"]
Sep 15 20:58:58.161404 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.161163    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-ddcc97c8b-72s7f"]
Sep 15 20:58:58.161404 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.161186    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d5cd9679-h825p"]
Sep 15 20:58:58.161404 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.161206    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-rlf4s"]
Sep 15 20:58:58.161404 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.161225    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9"]
Sep 15 20:58:58.161404 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.161245    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67dc6c86b6-pql25"]
Sep 15 20:58:58.162165 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.162147    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9"
Sep 15 20:58:58.165991 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.165970    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 15 20:58:58.166465 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.166446    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 15 20:58:58.167326 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.167310    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 15 20:58:58.167882 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.167863    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 15 20:58:58.183552 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.183528    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67dc6c86b6-pql25"]
Sep 15 20:58:58.183677 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.183572    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67dc6c86b6-pql25"
Sep 15 20:58:58.183677 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.183582    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-57cf586d6b-mkkv4"]
Sep 15 20:58:58.186721 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.186702    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 15 20:58:58.191389 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.191283    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-qrgq4\""
Sep 15 20:58:58.196916 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.196891    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ffff173-a298-4e73-98e3-d77f1c0a11e8-trusted-ca-bundle\") pod \"insights-operator-5844b856db-rlf4s\" (UID: \"4ffff173-a298-4e73-98e3-d77f1c0a11e8\") " pod="openshift-insights/insights-operator-5844b856db-rlf4s"
Sep 15 20:58:58.197003 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.196925    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d178fec0-ff06-43b0-a8c3-dd5e319d7141-tmp\") pod \"klusterlet-addon-workmgr-7d5cd9679-h825p\" (UID: \"d178fec0-ff06-43b0-a8c3-dd5e319d7141\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d5cd9679-h825p"
Sep 15 20:58:58.197003 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.196952    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d9d3b55f-e9eb-43c9-bc63-0b59e8d52cda-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-qlv8v\" (UID: \"d9d3b55f-e9eb-43c9-bc63-0b59e8d52cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qlv8v"
Sep 15 20:58:58.197003 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.196987    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhq4f\" (UniqueName: \"kubernetes.io/projected/d9d3b55f-e9eb-43c9-bc63-0b59e8d52cda-kube-api-access-zhq4f\") pod \"cluster-monitoring-operator-6cd954b897-qlv8v\" (UID: \"d9d3b55f-e9eb-43c9-bc63-0b59e8d52cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qlv8v"
Sep 15 20:58:58.197131 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.197029    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xmwd\" (UniqueName: \"kubernetes.io/projected/d178fec0-ff06-43b0-a8c3-dd5e319d7141-kube-api-access-2xmwd\") pod \"klusterlet-addon-workmgr-7d5cd9679-h825p\" (UID: \"d178fec0-ff06-43b0-a8c3-dd5e319d7141\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d5cd9679-h825p"
Sep 15 20:58:58.197131 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.197085    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ffff173-a298-4e73-98e3-d77f1c0a11e8-service-ca-bundle\") pod \"insights-operator-5844b856db-rlf4s\" (UID: \"4ffff173-a298-4e73-98e3-d77f1c0a11e8\") " pod="openshift-insights/insights-operator-5844b856db-rlf4s"
Sep 15 20:58:58.197131 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.197100    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4ffff173-a298-4e73-98e3-d77f1c0a11e8-serving-cert\") pod \"insights-operator-5844b856db-rlf4s\" (UID: \"4ffff173-a298-4e73-98e3-d77f1c0a11e8\") " pod="openshift-insights/insights-operator-5844b856db-rlf4s"
Sep 15 20:58:58.197131 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.197116    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b852r\" (UniqueName: \"kubernetes.io/projected/4ffff173-a298-4e73-98e3-d77f1c0a11e8-kube-api-access-b852r\") pod \"insights-operator-5844b856db-rlf4s\" (UID: \"4ffff173-a298-4e73-98e3-d77f1c0a11e8\") " pod="openshift-insights/insights-operator-5844b856db-rlf4s"
Sep 15 20:58:58.197302 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.197156    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ffff173-a298-4e73-98e3-d77f1c0a11e8-tmp\") pod \"insights-operator-5844b856db-rlf4s\" (UID: \"4ffff173-a298-4e73-98e3-d77f1c0a11e8\") " pod="openshift-insights/insights-operator-5844b856db-rlf4s"
Sep 15 20:58:58.197302 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.197234    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4ffff173-a298-4e73-98e3-d77f1c0a11e8-snapshots\") pod \"insights-operator-5844b856db-rlf4s\" (UID: \"4ffff173-a298-4e73-98e3-d77f1c0a11e8\") " pod="openshift-insights/insights-operator-5844b856db-rlf4s"
Sep 15 20:58:58.197302 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.197277    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d9d3b55f-e9eb-43c9-bc63-0b59e8d52cda-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-qlv8v\" (UID: \"d9d3b55f-e9eb-43c9-bc63-0b59e8d52cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qlv8v"
Sep 15 20:58:58.197418 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.197303    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d178fec0-ff06-43b0-a8c3-dd5e319d7141-klusterlet-config\") pod \"klusterlet-addon-workmgr-7d5cd9679-h825p\" (UID: \"d178fec0-ff06-43b0-a8c3-dd5e319d7141\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d5cd9679-h825p"
Sep 15 20:58:58.203459 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.203441    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-n5nqb"]
Sep 15 20:58:58.203613 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.203596    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.221433 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.221413    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-57cf586d6b-mkkv4"]
Sep 15 20:58:58.221562 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.221444    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-n5nqb"]
Sep 15 20:58:58.221562 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.221545    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-n5nqb"
Sep 15 20:58:58.225652 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.225634    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 15 20:58:58.226018 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.226000    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-k5rgw\""
Sep 15 20:58:58.228406 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.228391    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 15 20:58:58.250258 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.250227    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:58.298177 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298138    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/16a7c103-2e47-4956-8ba1-9d9fbd0a27e9-ca\") pod \"cluster-proxy-proxy-agent-66b6854579-dvwx9\" (UID: \"16a7c103-2e47-4956-8ba1-9d9fbd0a27e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9"
Sep 15 20:58:58.298364 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298189    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-registry-certificates\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.298364 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298215    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34f22718-53a1-41fd-8eda-f8581983bf3d-metrics-certs\") pod \"router-default-ddcc97c8b-72s7f\" (UID: \"34f22718-53a1-41fd-8eda-f8581983bf3d\") " pod="openshift-ingress/router-default-ddcc97c8b-72s7f"
Sep 15 20:58:58.298364 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298271    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/16a7c103-2e47-4956-8ba1-9d9fbd0a27e9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-66b6854579-dvwx9\" (UID: \"16a7c103-2e47-4956-8ba1-9d9fbd0a27e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9"
Sep 15 20:58:58.298364 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298317    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/319f43bf-8430-42a8-bf80-862f878cf486-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-67dc6c86b6-pql25\" (UID: \"319f43bf-8430-42a8-bf80-862f878cf486\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67dc6c86b6-pql25"
Sep 15 20:58:58.298364 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298344    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d178fec0-ff06-43b0-a8c3-dd5e319d7141-tmp\") pod \"klusterlet-addon-workmgr-7d5cd9679-h825p\" (UID: \"d178fec0-ff06-43b0-a8c3-dd5e319d7141\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d5cd9679-h825p"
Sep 15 20:58:58.298364 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298364    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d9d3b55f-e9eb-43c9-bc63-0b59e8d52cda-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-qlv8v\" (UID: \"d9d3b55f-e9eb-43c9-bc63-0b59e8d52cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qlv8v"
Sep 15 20:58:58.298773 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298389    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhq4f\" (UniqueName: \"kubernetes.io/projected/d9d3b55f-e9eb-43c9-bc63-0b59e8d52cda-kube-api-access-zhq4f\") pod \"cluster-monitoring-operator-6cd954b897-qlv8v\" (UID: \"d9d3b55f-e9eb-43c9-bc63-0b59e8d52cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qlv8v"
Sep 15 20:58:58.298773 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298485    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xmwd\" (UniqueName: \"kubernetes.io/projected/d178fec0-ff06-43b0-a8c3-dd5e319d7141-kube-api-access-2xmwd\") pod \"klusterlet-addon-workmgr-7d5cd9679-h825p\" (UID: \"d178fec0-ff06-43b0-a8c3-dd5e319d7141\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d5cd9679-h825p"
Sep 15 20:58:58.298773 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298549    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ffff173-a298-4e73-98e3-d77f1c0a11e8-service-ca-bundle\") pod \"insights-operator-5844b856db-rlf4s\" (UID: \"4ffff173-a298-4e73-98e3-d77f1c0a11e8\") " pod="openshift-insights/insights-operator-5844b856db-rlf4s"
Sep 15 20:58:58.298773 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298574    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4ffff173-a298-4e73-98e3-d77f1c0a11e8-serving-cert\") pod \"insights-operator-5844b856db-rlf4s\" (UID: \"4ffff173-a298-4e73-98e3-d77f1c0a11e8\") " pod="openshift-insights/insights-operator-5844b856db-rlf4s"
Sep 15 20:58:58.298773 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298599    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b852r\" (UniqueName: \"kubernetes.io/projected/4ffff173-a298-4e73-98e3-d77f1c0a11e8-kube-api-access-b852r\") pod \"insights-operator-5844b856db-rlf4s\" (UID: \"4ffff173-a298-4e73-98e3-d77f1c0a11e8\") " pod="openshift-insights/insights-operator-5844b856db-rlf4s"
Sep 15 20:58:58.298773 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298629    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ffff173-a298-4e73-98e3-d77f1c0a11e8-tmp\") pod \"insights-operator-5844b856db-rlf4s\" (UID: \"4ffff173-a298-4e73-98e3-d77f1c0a11e8\") " pod="openshift-insights/insights-operator-5844b856db-rlf4s"
Sep 15 20:58:58.298773 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298654    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ll75\" (UniqueName: \"kubernetes.io/projected/319f43bf-8430-42a8-bf80-862f878cf486-kube-api-access-6ll75\") pod \"managed-serviceaccount-addon-agent-67dc6c86b6-pql25\" (UID: \"319f43bf-8430-42a8-bf80-862f878cf486\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67dc6c86b6-pql25"
Sep 15 20:58:58.298773 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298699    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-bound-sa-token\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.298773 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298734    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34f22718-53a1-41fd-8eda-f8581983bf3d-service-ca-bundle\") pod \"router-default-ddcc97c8b-72s7f\" (UID: \"34f22718-53a1-41fd-8eda-f8581983bf3d\") " pod="openshift-ingress/router-default-ddcc97c8b-72s7f"
Sep 15 20:58:58.298773 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298766    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-registry-tls\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.299281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298809    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/34f22718-53a1-41fd-8eda-f8581983bf3d-stats-auth\") pod \"router-default-ddcc97c8b-72s7f\" (UID: \"34f22718-53a1-41fd-8eda-f8581983bf3d\") " pod="openshift-ingress/router-default-ddcc97c8b-72s7f"
Sep 15 20:58:58.299281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298850    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d178fec0-ff06-43b0-a8c3-dd5e319d7141-tmp\") pod \"klusterlet-addon-workmgr-7d5cd9679-h825p\" (UID: \"d178fec0-ff06-43b0-a8c3-dd5e319d7141\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d5cd9679-h825p"
Sep 15 20:58:58.299281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298854    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4ffff173-a298-4e73-98e3-d77f1c0a11e8-snapshots\") pod \"insights-operator-5844b856db-rlf4s\" (UID: \"4ffff173-a298-4e73-98e3-d77f1c0a11e8\") " pod="openshift-insights/insights-operator-5844b856db-rlf4s"
Sep 15 20:58:58.299281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298892    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/16a7c103-2e47-4956-8ba1-9d9fbd0a27e9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-66b6854579-dvwx9\" (UID: \"16a7c103-2e47-4956-8ba1-9d9fbd0a27e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9"
Sep 15 20:58:58.299281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298929    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2thcs\" (UniqueName: \"kubernetes.io/projected/5c896e08-2021-4261-8701-ffb73d0bbed9-kube-api-access-2thcs\") pod \"downloads-76f99584f4-n5nqb\" (UID: \"5c896e08-2021-4261-8701-ffb73d0bbed9\") " pod="openshift-console/downloads-76f99584f4-n5nqb"
Sep 15 20:58:58.299281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298954    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqhzx\" (UniqueName: \"kubernetes.io/projected/34f22718-53a1-41fd-8eda-f8581983bf3d-kube-api-access-pqhzx\") pod \"router-default-ddcc97c8b-72s7f\" (UID: \"34f22718-53a1-41fd-8eda-f8581983bf3d\") " pod="openshift-ingress/router-default-ddcc97c8b-72s7f"
Sep 15 20:58:58.299281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.298984    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d9d3b55f-e9eb-43c9-bc63-0b59e8d52cda-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-qlv8v\" (UID: \"d9d3b55f-e9eb-43c9-bc63-0b59e8d52cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qlv8v"
Sep 15 20:58:58.299281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.299006    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d178fec0-ff06-43b0-a8c3-dd5e319d7141-klusterlet-config\") pod \"klusterlet-addon-workmgr-7d5cd9679-h825p\" (UID: \"d178fec0-ff06-43b0-a8c3-dd5e319d7141\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d5cd9679-h825p"
Sep 15 20:58:58.299281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.299030    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-image-registry-private-configuration\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.299281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.299051    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/16a7c103-2e47-4956-8ba1-9d9fbd0a27e9-hub\") pod \"cluster-proxy-proxy-agent-66b6854579-dvwx9\" (UID: \"16a7c103-2e47-4956-8ba1-9d9fbd0a27e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9"
Sep 15 20:58:58.299281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.299073    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kppch\" (UniqueName: \"kubernetes.io/projected/16a7c103-2e47-4956-8ba1-9d9fbd0a27e9-kube-api-access-kppch\") pod \"cluster-proxy-proxy-agent-66b6854579-dvwx9\" (UID: \"16a7c103-2e47-4956-8ba1-9d9fbd0a27e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9"
Sep 15 20:58:58.299281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.299100    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqq64\" (UniqueName: \"kubernetes.io/projected/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-kube-api-access-nqq64\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.299281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.299102    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ffff173-a298-4e73-98e3-d77f1c0a11e8-tmp\") pod \"insights-operator-5844b856db-rlf4s\" (UID: \"4ffff173-a298-4e73-98e3-d77f1c0a11e8\") " pod="openshift-insights/insights-operator-5844b856db-rlf4s"
Sep 15 20:58:58.299281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.299124    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-installation-pull-secrets\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.299281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.299151    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-ca-trust-extracted\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.299884 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.299199    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ffff173-a298-4e73-98e3-d77f1c0a11e8-trusted-ca-bundle\") pod \"insights-operator-5844b856db-rlf4s\" (UID: \"4ffff173-a298-4e73-98e3-d77f1c0a11e8\") " pod="openshift-insights/insights-operator-5844b856db-rlf4s"
Sep 15 20:58:58.299884 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.299223    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-trusted-ca\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.299884 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.299222    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d9d3b55f-e9eb-43c9-bc63-0b59e8d52cda-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-qlv8v\" (UID: \"d9d3b55f-e9eb-43c9-bc63-0b59e8d52cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qlv8v"
Sep 15 20:58:58.299884 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.299247    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/34f22718-53a1-41fd-8eda-f8581983bf3d-default-certificate\") pod \"router-default-ddcc97c8b-72s7f\" (UID: \"34f22718-53a1-41fd-8eda-f8581983bf3d\") " pod="openshift-ingress/router-default-ddcc97c8b-72s7f"
Sep 15 20:58:58.299884 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.299287    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/16a7c103-2e47-4956-8ba1-9d9fbd0a27e9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-66b6854579-dvwx9\" (UID: \"16a7c103-2e47-4956-8ba1-9d9fbd0a27e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9"
Sep 15 20:58:58.299884 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.299340    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ffff173-a298-4e73-98e3-d77f1c0a11e8-service-ca-bundle\") pod \"insights-operator-5844b856db-rlf4s\" (UID: \"4ffff173-a298-4e73-98e3-d77f1c0a11e8\") " pod="openshift-insights/insights-operator-5844b856db-rlf4s"
Sep 15 20:58:58.299884 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.299442    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4ffff173-a298-4e73-98e3-d77f1c0a11e8-snapshots\") pod \"insights-operator-5844b856db-rlf4s\" (UID: \"4ffff173-a298-4e73-98e3-d77f1c0a11e8\") " pod="openshift-insights/insights-operator-5844b856db-rlf4s"
Sep 15 20:58:58.300149 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.299891    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ffff173-a298-4e73-98e3-d77f1c0a11e8-trusted-ca-bundle\") pod \"insights-operator-5844b856db-rlf4s\" (UID: \"4ffff173-a298-4e73-98e3-d77f1c0a11e8\") " pod="openshift-insights/insights-operator-5844b856db-rlf4s"
Sep 15 20:58:58.301368 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.301341    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4ffff173-a298-4e73-98e3-d77f1c0a11e8-serving-cert\") pod \"insights-operator-5844b856db-rlf4s\" (UID: \"4ffff173-a298-4e73-98e3-d77f1c0a11e8\") " pod="openshift-insights/insights-operator-5844b856db-rlf4s"
Sep 15 20:58:58.302234 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.302206    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d9d3b55f-e9eb-43c9-bc63-0b59e8d52cda-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-qlv8v\" (UID: \"d9d3b55f-e9eb-43c9-bc63-0b59e8d52cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qlv8v"
Sep 15 20:58:58.302322 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.302285    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d178fec0-ff06-43b0-a8c3-dd5e319d7141-klusterlet-config\") pod \"klusterlet-addon-workmgr-7d5cd9679-h825p\" (UID: \"d178fec0-ff06-43b0-a8c3-dd5e319d7141\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d5cd9679-h825p"
Sep 15 20:58:58.314298 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.314262    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xmwd\" (UniqueName: \"kubernetes.io/projected/d178fec0-ff06-43b0-a8c3-dd5e319d7141-kube-api-access-2xmwd\") pod \"klusterlet-addon-workmgr-7d5cd9679-h825p\" (UID: \"d178fec0-ff06-43b0-a8c3-dd5e319d7141\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d5cd9679-h825p"
Sep 15 20:58:58.314721 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.314702    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhq4f\" (UniqueName: \"kubernetes.io/projected/d9d3b55f-e9eb-43c9-bc63-0b59e8d52cda-kube-api-access-zhq4f\") pod \"cluster-monitoring-operator-6cd954b897-qlv8v\" (UID: \"d9d3b55f-e9eb-43c9-bc63-0b59e8d52cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qlv8v"
Sep 15 20:58:58.315396 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.315378    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b852r\" (UniqueName: \"kubernetes.io/projected/4ffff173-a298-4e73-98e3-d77f1c0a11e8-kube-api-access-b852r\") pod \"insights-operator-5844b856db-rlf4s\" (UID: \"4ffff173-a298-4e73-98e3-d77f1c0a11e8\") " pod="openshift-insights/insights-operator-5844b856db-rlf4s"
Sep 15 20:58:58.336999 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.336961    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:58.392829 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.392795    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-rlf4s"
Sep 15 20:58:58.400073 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.400035    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ll75\" (UniqueName: \"kubernetes.io/projected/319f43bf-8430-42a8-bf80-862f878cf486-kube-api-access-6ll75\") pod \"managed-serviceaccount-addon-agent-67dc6c86b6-pql25\" (UID: \"319f43bf-8430-42a8-bf80-862f878cf486\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67dc6c86b6-pql25"
Sep 15 20:58:58.400237 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.400079    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-bound-sa-token\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.400237 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.400102    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34f22718-53a1-41fd-8eda-f8581983bf3d-service-ca-bundle\") pod \"router-default-ddcc97c8b-72s7f\" (UID: \"34f22718-53a1-41fd-8eda-f8581983bf3d\") " pod="openshift-ingress/router-default-ddcc97c8b-72s7f"
Sep 15 20:58:58.400237 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.400134    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-registry-tls\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.400237 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.400159    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/34f22718-53a1-41fd-8eda-f8581983bf3d-stats-auth\") pod \"router-default-ddcc97c8b-72s7f\" (UID: \"34f22718-53a1-41fd-8eda-f8581983bf3d\") " pod="openshift-ingress/router-default-ddcc97c8b-72s7f"
Sep 15 20:58:58.400237 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.400186    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/16a7c103-2e47-4956-8ba1-9d9fbd0a27e9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-66b6854579-dvwx9\" (UID: \"16a7c103-2e47-4956-8ba1-9d9fbd0a27e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9"
Sep 15 20:58:58.400237 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.400210    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2thcs\" (UniqueName: \"kubernetes.io/projected/5c896e08-2021-4261-8701-ffb73d0bbed9-kube-api-access-2thcs\") pod \"downloads-76f99584f4-n5nqb\" (UID: \"5c896e08-2021-4261-8701-ffb73d0bbed9\") " pod="openshift-console/downloads-76f99584f4-n5nqb"
Sep 15 20:58:58.400237 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.400234    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqhzx\" (UniqueName: \"kubernetes.io/projected/34f22718-53a1-41fd-8eda-f8581983bf3d-kube-api-access-pqhzx\") pod \"router-default-ddcc97c8b-72s7f\" (UID: \"34f22718-53a1-41fd-8eda-f8581983bf3d\") " pod="openshift-ingress/router-default-ddcc97c8b-72s7f"
Sep 15 20:58:58.400658 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.400264    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-image-registry-private-configuration\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.400658 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.400286    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/16a7c103-2e47-4956-8ba1-9d9fbd0a27e9-hub\") pod \"cluster-proxy-proxy-agent-66b6854579-dvwx9\" (UID: \"16a7c103-2e47-4956-8ba1-9d9fbd0a27e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9"
Sep 15 20:58:58.400658 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.400308    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kppch\" (UniqueName: \"kubernetes.io/projected/16a7c103-2e47-4956-8ba1-9d9fbd0a27e9-kube-api-access-kppch\") pod \"cluster-proxy-proxy-agent-66b6854579-dvwx9\" (UID: \"16a7c103-2e47-4956-8ba1-9d9fbd0a27e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9"
Sep 15 20:58:58.400658 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.400339    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqq64\" (UniqueName: \"kubernetes.io/projected/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-kube-api-access-nqq64\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.400658 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.400365    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-installation-pull-secrets\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.400658 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.400391    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-ca-trust-extracted\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.400658 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.400433    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-trusted-ca\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.400658 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.400455    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/34f22718-53a1-41fd-8eda-f8581983bf3d-default-certificate\") pod \"router-default-ddcc97c8b-72s7f\" (UID: \"34f22718-53a1-41fd-8eda-f8581983bf3d\") " pod="openshift-ingress/router-default-ddcc97c8b-72s7f"
Sep 15 20:58:58.400658 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.400483    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/16a7c103-2e47-4956-8ba1-9d9fbd0a27e9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-66b6854579-dvwx9\" (UID: \"16a7c103-2e47-4956-8ba1-9d9fbd0a27e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9"
Sep 15 20:58:58.400658 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.400552    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/16a7c103-2e47-4956-8ba1-9d9fbd0a27e9-ca\") pod \"cluster-proxy-proxy-agent-66b6854579-dvwx9\" (UID: \"16a7c103-2e47-4956-8ba1-9d9fbd0a27e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9"
Sep 15 20:58:58.400658 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.400583    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-registry-certificates\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.400658 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.400606    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34f22718-53a1-41fd-8eda-f8581983bf3d-metrics-certs\") pod \"router-default-ddcc97c8b-72s7f\" (UID: \"34f22718-53a1-41fd-8eda-f8581983bf3d\") " pod="openshift-ingress/router-default-ddcc97c8b-72s7f"
Sep 15 20:58:58.400658 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.400642    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/16a7c103-2e47-4956-8ba1-9d9fbd0a27e9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-66b6854579-dvwx9\" (UID: \"16a7c103-2e47-4956-8ba1-9d9fbd0a27e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9"
Sep 15 20:58:58.401474 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.400671    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/319f43bf-8430-42a8-bf80-862f878cf486-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-67dc6c86b6-pql25\" (UID: \"319f43bf-8430-42a8-bf80-862f878cf486\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67dc6c86b6-pql25"
Sep 15 20:58:58.401474 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.400946    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34f22718-53a1-41fd-8eda-f8581983bf3d-service-ca-bundle\") pod \"router-default-ddcc97c8b-72s7f\" (UID: \"34f22718-53a1-41fd-8eda-f8581983bf3d\") " pod="openshift-ingress/router-default-ddcc97c8b-72s7f"
Sep 15 20:58:58.403311 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.402555    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-ca-trust-extracted\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.404229 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.403698    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-registry-certificates\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.404229 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.403728    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-trusted-ca\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.404229 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.403732    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/16a7c103-2e47-4956-8ba1-9d9fbd0a27e9-hub\") pod \"cluster-proxy-proxy-agent-66b6854579-dvwx9\" (UID: \"16a7c103-2e47-4956-8ba1-9d9fbd0a27e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9"
Sep 15 20:58:58.404229 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.404188    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-registry-tls\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.404229 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.404195    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/34f22718-53a1-41fd-8eda-f8581983bf3d-stats-auth\") pod \"router-default-ddcc97c8b-72s7f\" (UID: \"34f22718-53a1-41fd-8eda-f8581983bf3d\") " pod="openshift-ingress/router-default-ddcc97c8b-72s7f"
Sep 15 20:58:58.405076 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.404680    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/16a7c103-2e47-4956-8ba1-9d9fbd0a27e9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-66b6854579-dvwx9\" (UID: \"16a7c103-2e47-4956-8ba1-9d9fbd0a27e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9"
Sep 15 20:58:58.405076 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.405040    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/34f22718-53a1-41fd-8eda-f8581983bf3d-default-certificate\") pod \"router-default-ddcc97c8b-72s7f\" (UID: \"34f22718-53a1-41fd-8eda-f8581983bf3d\") " pod="openshift-ingress/router-default-ddcc97c8b-72s7f"
Sep 15 20:58:58.405661 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.405612    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/319f43bf-8430-42a8-bf80-862f878cf486-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-67dc6c86b6-pql25\" (UID: \"319f43bf-8430-42a8-bf80-862f878cf486\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67dc6c86b6-pql25"
Sep 15 20:58:58.406165 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.406137    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/16a7c103-2e47-4956-8ba1-9d9fbd0a27e9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-66b6854579-dvwx9\" (UID: \"16a7c103-2e47-4956-8ba1-9d9fbd0a27e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9"
Sep 15 20:58:58.406355 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.406327    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-image-registry-private-configuration\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.406440 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.406327    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/16a7c103-2e47-4956-8ba1-9d9fbd0a27e9-ca\") pod \"cluster-proxy-proxy-agent-66b6854579-dvwx9\" (UID: \"16a7c103-2e47-4956-8ba1-9d9fbd0a27e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9"
Sep 15 20:58:58.406440 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.406331    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/16a7c103-2e47-4956-8ba1-9d9fbd0a27e9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-66b6854579-dvwx9\" (UID: \"16a7c103-2e47-4956-8ba1-9d9fbd0a27e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9"
Sep 15 20:58:58.406729 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.406709    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34f22718-53a1-41fd-8eda-f8581983bf3d-metrics-certs\") pod \"router-default-ddcc97c8b-72s7f\" (UID: \"34f22718-53a1-41fd-8eda-f8581983bf3d\") " pod="openshift-ingress/router-default-ddcc97c8b-72s7f"
Sep 15 20:58:58.407243 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.407219    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-installation-pull-secrets\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.410705 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.410686    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qlv8v"
Sep 15 20:58:58.411813 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.411793    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-bound-sa-token\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.417145 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.417123    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqhzx\" (UniqueName: \"kubernetes.io/projected/34f22718-53a1-41fd-8eda-f8581983bf3d-kube-api-access-pqhzx\") pod \"router-default-ddcc97c8b-72s7f\" (UID: \"34f22718-53a1-41fd-8eda-f8581983bf3d\") " pod="openshift-ingress/router-default-ddcc97c8b-72s7f"
Sep 15 20:58:58.417622 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.417592    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kppch\" (UniqueName: \"kubernetes.io/projected/16a7c103-2e47-4956-8ba1-9d9fbd0a27e9-kube-api-access-kppch\") pod \"cluster-proxy-proxy-agent-66b6854579-dvwx9\" (UID: \"16a7c103-2e47-4956-8ba1-9d9fbd0a27e9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9"
Sep 15 20:58:58.417752 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.417731    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2thcs\" (UniqueName: \"kubernetes.io/projected/5c896e08-2021-4261-8701-ffb73d0bbed9-kube-api-access-2thcs\") pod \"downloads-76f99584f4-n5nqb\" (UID: \"5c896e08-2021-4261-8701-ffb73d0bbed9\") " pod="openshift-console/downloads-76f99584f4-n5nqb"
Sep 15 20:58:58.418089 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.418064    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ll75\" (UniqueName: \"kubernetes.io/projected/319f43bf-8430-42a8-bf80-862f878cf486-kube-api-access-6ll75\") pod \"managed-serviceaccount-addon-agent-67dc6c86b6-pql25\" (UID: \"319f43bf-8430-42a8-bf80-862f878cf486\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67dc6c86b6-pql25"
Sep 15 20:58:58.418089 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.418085    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqq64\" (UniqueName: \"kubernetes.io/projected/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-kube-api-access-nqq64\") pod \"image-registry-57cf586d6b-mkkv4\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") " pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.430724 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.430698    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d5cd9679-h825p"
Sep 15 20:58:58.436914 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.436889    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:58.452718 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.452685    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-ddcc97c8b-72s7f"
Sep 15 20:58:58.477245 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.477026    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9"
Sep 15 20:58:58.492347 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.491827    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67dc6c86b6-pql25"
Sep 15 20:58:58.535034 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.534904    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:58.538761 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.538729    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:58.538974 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.538955    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-n5nqb"
Sep 15 20:58:58.639249 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.639208    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:58:58.701000 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.700973    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-rlf4s"]
Sep 15 20:58:58.703916 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:58:58.703889    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ffff173_a298_4e73_98e3_d77f1c0a11e8.slice/crio-1ac8cd2e10dfac39bc8e49a5033380e213306550585da5357af21489ed3b2e40 WatchSource:0}: Error finding container 1ac8cd2e10dfac39bc8e49a5033380e213306550585da5357af21489ed3b2e40: Status 404 returned error can't find the container with id 1ac8cd2e10dfac39bc8e49a5033380e213306550585da5357af21489ed3b2e40
Sep 15 20:58:58.743205 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.743175    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-qlv8v"]
Sep 15 20:58:58.745466 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:58:58.745441    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9d3b55f_e9eb_43c9_bc63_0b59e8d52cda.slice/crio-01e434483aec7cabcd648555961c8aef5a13d7f3f423067daf0e117aaf2a9888 WatchSource:0}: Error finding container 01e434483aec7cabcd648555961c8aef5a13d7f3f423067daf0e117aaf2a9888: Status 404 returned error can't find the container with id 01e434483aec7cabcd648555961c8aef5a13d7f3f423067daf0e117aaf2a9888
Sep 15 20:58:58.772141 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.772117    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d5cd9679-h825p"]
Sep 15 20:58:58.774311 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:58:58.774286    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd178fec0_ff06_43b0_a8c3_dd5e319d7141.slice/crio-66f5c4def56980933fe229f7b12b295a190c309d2ab703e28277dc3d3e983c71 WatchSource:0}: Error finding container 66f5c4def56980933fe229f7b12b295a190c309d2ab703e28277dc3d3e983c71: Status 404 returned error can't find the container with id 66f5c4def56980933fe229f7b12b295a190c309d2ab703e28277dc3d3e983c71
Sep 15 20:58:58.802461 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.802400    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9"]
Sep 15 20:58:58.805129 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:58:58.805107    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16a7c103_2e47_4956_8ba1_9d9fbd0a27e9.slice/crio-e98492f4ee71fa7d3cf5a9d366891bba53211dab05b507a7571986fd15db6963 WatchSource:0}: Error finding container e98492f4ee71fa7d3cf5a9d366891bba53211dab05b507a7571986fd15db6963: Status 404 returned error can't find the container with id e98492f4ee71fa7d3cf5a9d366891bba53211dab05b507a7571986fd15db6963
Sep 15 20:58:58.837484 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.837447    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-ddcc97c8b-72s7f"]
Sep 15 20:58:58.840083 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:58:58.840062    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34f22718_53a1_41fd_8eda_f8581983bf3d.slice/crio-2d7c3dce2d35bf031cf395641ad5fe076aba64320cb5367871b4243a5b2d27f9 WatchSource:0}: Error finding container 2d7c3dce2d35bf031cf395641ad5fe076aba64320cb5367871b4243a5b2d27f9: Status 404 returned error can't find the container with id 2d7c3dce2d35bf031cf395641ad5fe076aba64320cb5367871b4243a5b2d27f9
Sep 15 20:58:58.865949 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.865920    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67dc6c86b6-pql25"]
Sep 15 20:58:58.868151 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:58:58.868121    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod319f43bf_8430_42a8_bf80_862f878cf486.slice/crio-3aaa247ad31395aa3c91f3a01ceee1e8739c8cea414723285b1f447c402a506e WatchSource:0}: Error finding container 3aaa247ad31395aa3c91f3a01ceee1e8739c8cea414723285b1f447c402a506e: Status 404 returned error can't find the container with id 3aaa247ad31395aa3c91f3a01ceee1e8739c8cea414723285b1f447c402a506e
Sep 15 20:58:58.869381 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.869358    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-n5nqb"]
Sep 15 20:58:58.870915 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:58:58.870891    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c896e08_2021_4261_8701_ffb73d0bbed9.slice/crio-c0a5fe50b292b9890fa491a5ba9d61eb3b74664c377a3d79f11bae2e341095df WatchSource:0}: Error finding container c0a5fe50b292b9890fa491a5ba9d61eb3b74664c377a3d79f11bae2e341095df: Status 404 returned error can't find the container with id c0a5fe50b292b9890fa491a5ba9d61eb3b74664c377a3d79f11bae2e341095df
Sep 15 20:58:58.906590 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:58.906553    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-57cf586d6b-mkkv4"]
Sep 15 20:58:58.909788 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:58:58.909662    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbbd33a2_7ea1_4344_ab0e_ef5e6494bc6e.slice/crio-b3f4220f18ca292b941539f607e788ecbd5a1bd4f87c182e1e10f56db087a81c WatchSource:0}: Error finding container b3f4220f18ca292b941539f607e788ecbd5a1bd4f87c182e1e10f56db087a81c: Status 404 returned error can't find the container with id b3f4220f18ca292b941539f607e788ecbd5a1bd4f87c182e1e10f56db087a81c
Sep 15 20:58:59.464461 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:59.463899    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-ddcc97c8b-72s7f" event={"ID":"34f22718-53a1-41fd-8eda-f8581983bf3d","Type":"ContainerStarted","Data":"964163791b478da18a0c679666d6a9c668b1f45e4c2d97d31309f765fca43118"}
Sep 15 20:58:59.464461 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:59.463971    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-ddcc97c8b-72s7f" event={"ID":"34f22718-53a1-41fd-8eda-f8581983bf3d","Type":"ContainerStarted","Data":"2d7c3dce2d35bf031cf395641ad5fe076aba64320cb5367871b4243a5b2d27f9"}
Sep 15 20:58:59.465456 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:59.465422    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-n5nqb" event={"ID":"5c896e08-2021-4261-8701-ffb73d0bbed9","Type":"ContainerStarted","Data":"c0a5fe50b292b9890fa491a5ba9d61eb3b74664c377a3d79f11bae2e341095df"}
Sep 15 20:58:59.466810 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:59.466785    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-rlf4s" event={"ID":"4ffff173-a298-4e73-98e3-d77f1c0a11e8","Type":"ContainerStarted","Data":"1ac8cd2e10dfac39bc8e49a5033380e213306550585da5357af21489ed3b2e40"}
Sep 15 20:58:59.468146 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:59.468123    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qlv8v" event={"ID":"d9d3b55f-e9eb-43c9-bc63-0b59e8d52cda","Type":"ContainerStarted","Data":"01e434483aec7cabcd648555961c8aef5a13d7f3f423067daf0e117aaf2a9888"}
Sep 15 20:58:59.469940 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:59.469912    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4" event={"ID":"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e","Type":"ContainerStarted","Data":"132eae7c64a215fc85da0d0543892f7c75d187907c6adec7bc6232fa2e52708c"}
Sep 15 20:58:59.470069 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:59.469955    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4" event={"ID":"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e","Type":"ContainerStarted","Data":"b3f4220f18ca292b941539f607e788ecbd5a1bd4f87c182e1e10f56db087a81c"}
Sep 15 20:58:59.470069 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:59.470011    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:58:59.471205 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:59.471185    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9" event={"ID":"16a7c103-2e47-4956-8ba1-9d9fbd0a27e9","Type":"ContainerStarted","Data":"e98492f4ee71fa7d3cf5a9d366891bba53211dab05b507a7571986fd15db6963"}
Sep 15 20:58:59.472473 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:59.472448    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d5cd9679-h825p" event={"ID":"d178fec0-ff06-43b0-a8c3-dd5e319d7141","Type":"ContainerStarted","Data":"66f5c4def56980933fe229f7b12b295a190c309d2ab703e28277dc3d3e983c71"}
Sep 15 20:58:59.474138 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:59.474118    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67dc6c86b6-pql25" event={"ID":"319f43bf-8430-42a8-bf80-862f878cf486","Type":"ContainerStarted","Data":"3aaa247ad31395aa3c91f3a01ceee1e8739c8cea414723285b1f447c402a506e"}
Sep 15 20:58:59.483109 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:59.483058    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-ddcc97c8b-72s7f" podStartSLOduration=1.483037365 podStartE2EDuration="1.483037365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:58:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:59.480773386 +0000 UTC m=+76.022936048" watchObservedRunningTime="2026-09-15 20:58:59.483037365 +0000 UTC m=+76.025199995"
Sep 15 20:58:59.500079 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:58:59.500033    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4" podStartSLOduration=68.500015944 podStartE2EDuration="1m8.500015944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:57:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:58:59.498676718 +0000 UTC m=+76.040839379" watchObservedRunningTime="2026-09-15 20:58:59.500015944 +0000 UTC m=+76.042178569"
Sep 15 20:59:00.455597 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:00.455561    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-ddcc97c8b-72s7f"
Sep 15 20:59:00.458195 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:00.458167    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-ddcc97c8b-72s7f"
Sep 15 20:59:00.483473 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:00.483422    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-ddcc97c8b-72s7f"
Sep 15 20:59:00.485285 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:00.485072    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-ddcc97c8b-72s7f"
Sep 15 20:59:01.318144 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:01.318110    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-m8fpw"
Sep 15 20:59:02.111780 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:02.111732    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:02.367924 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:02.367800    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:02.437127 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:02.437080    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:03.192385 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:03.192343    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:03.296007 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:03.295958    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:03.445052 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:03.444963    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rqmrf"
Sep 15 20:59:04.693025 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:04.692348    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:04.790153 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:04.790021    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:06.900282 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:06.900241    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:07.001272 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:07.001214    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:07.318129 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:07.318040    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:07.419196 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:07.419104    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:08.170414 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.170383    2564 patch_prober.go:28] interesting pod/image-registry-846b7dcc97-fpgwt container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 15 20:59:08.170783 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.170438    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt" podUID="2243bd69-2640-4dc6-beaa-7e22b414db60" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 20:59:08.474611 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.474567    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:08.525723 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.525430    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-rlf4s" event={"ID":"4ffff173-a298-4e73-98e3-d77f1c0a11e8","Type":"ContainerStarted","Data":"e8fa71dc2301003c1a7b5f58837318359ca549cc1ef47fe83ae8015c0a0fb875"}
Sep 15 20:59:08.527703 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.527671    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qlv8v" event={"ID":"d9d3b55f-e9eb-43c9-bc63-0b59e8d52cda","Type":"ContainerStarted","Data":"67473dde6b8d89871d48724a2ed3c5bbbb39eacd94ae88158934fc4b95730898"}
Sep 15 20:59:08.531351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.530819    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9" event={"ID":"16a7c103-2e47-4956-8ba1-9d9fbd0a27e9","Type":"ContainerStarted","Data":"3f204607273205b285547c9b63a528f8e6046bfa1362a571d7951b71f2be8c93"}
Sep 15 20:59:08.531351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.530858    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9" event={"ID":"16a7c103-2e47-4956-8ba1-9d9fbd0a27e9","Type":"ContainerStarted","Data":"3a42e544e7bd2f34be59b5ae1b4a29b7b268887c5b9440e0e69e95f97b61cb0d"}
Sep 15 20:59:08.531351 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.530878    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9" event={"ID":"16a7c103-2e47-4956-8ba1-9d9fbd0a27e9","Type":"ContainerStarted","Data":"ae6b761d8302f7cfcb84d34562ec2dd912384efc9fddc85042e2102884fe031e"}
Sep 15 20:59:08.533281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.533255    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d5cd9679-h825p" event={"ID":"d178fec0-ff06-43b0-a8c3-dd5e319d7141","Type":"ContainerStarted","Data":"2dd868478f4ee0b3a89b78ed37cd43d3648705de3fe0d506a0820044e88ec669"}
Sep 15 20:59:08.533402 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.533309    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d5cd9679-h825p"
Sep 15 20:59:08.535591 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.535561    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d5cd9679-h825p"
Sep 15 20:59:08.535736 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.535716    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67dc6c86b6-pql25" event={"ID":"319f43bf-8430-42a8-bf80-862f878cf486","Type":"ContainerStarted","Data":"d0e602b3f38201884d67923f48b9720543c0f576c077c9128e31f1a889337b80"}
Sep 15 20:59:08.553082 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.553045    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:08.567531 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.567424    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-rlf4s" podStartSLOduration=1.265506406 podStartE2EDuration="10.567399614s" totalImagesPullingTime="9.301893208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:58:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:08.543854983 +0000 UTC m=+85.086017618" watchObservedRunningTime="2026-09-15 20:59:08.567399614 +0000 UTC m=+85.109562250"
Sep 15 20:59:08.568711 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.568130    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-66b6854579-dvwx9" podStartSLOduration=1.327567507 podStartE2EDuration="10.568115103s" totalImagesPullingTime="9.240547596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:58:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:08.566022353 +0000 UTC m=+85.108184999" watchObservedRunningTime="2026-09-15 20:59:08.568115103 +0000 UTC m=+85.110277735"
Sep 15 20:59:08.583780 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.582301    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67dc6c86b6-pql25" podStartSLOduration=1.40540639 podStartE2EDuration="10.582279441s" totalImagesPullingTime="9.176873051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:58:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:08.581125279 +0000 UTC m=+85.123287915" watchObservedRunningTime="2026-09-15 20:59:08.582279441 +0000 UTC m=+85.124442076"
Sep 15 20:59:08.601610 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.600807    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-qlv8v" podStartSLOduration=1.334487204 podStartE2EDuration="10.600785847s" totalImagesPullingTime="9.266298643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:58:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:08.599789938 +0000 UTC m=+85.141952595" watchObservedRunningTime="2026-09-15 20:59:08.600785847 +0000 UTC m=+85.142948484"
Sep 15 20:59:08.625300 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.623513    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d5cd9679-h825p" podStartSLOduration=1.338330008 podStartE2EDuration="10.62346962s" totalImagesPullingTime="9.285139612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:58:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:08.622894137 +0000 UTC m=+85.165056799" watchObservedRunningTime="2026-09-15 20:59:08.62346962 +0000 UTC m=+85.165632256"
Sep 15 20:59:08.657704 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.656549    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:08.914684 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.914648    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xc4sd"]
Sep 15 20:59:08.917759 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.917737    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:08.920728 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.920702    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 15 20:59:08.920862 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.920702    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 15 20:59:08.920862 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.920742    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-22kzx\""
Sep 15 20:59:08.927676 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.927641    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xc4sd"]
Sep 15 20:59:08.933136 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.933106    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:08.998794 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:08.998727    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:09.107734 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:09.107690    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8lnm\" (UniqueName: \"kubernetes.io/projected/8cba866d-6b74-4efa-802d-3d4248a87202-kube-api-access-j8lnm\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:09.107734 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:09.107737    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8cba866d-6b74-4efa-802d-3d4248a87202-data-volume\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:09.107974 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:09.107863    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8cba866d-6b74-4efa-802d-3d4248a87202-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:09.107974 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:09.107903    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cba866d-6b74-4efa-802d-3d4248a87202-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:09.107974 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:09.107957    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8cba866d-6b74-4efa-802d-3d4248a87202-crio-socket\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:09.209241 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:09.209155    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8lnm\" (UniqueName: \"kubernetes.io/projected/8cba866d-6b74-4efa-802d-3d4248a87202-kube-api-access-j8lnm\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:09.209241 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:09.209206    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8cba866d-6b74-4efa-802d-3d4248a87202-data-volume\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:09.209751 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:09.209255    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8cba866d-6b74-4efa-802d-3d4248a87202-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:09.209751 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:09.209276    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cba866d-6b74-4efa-802d-3d4248a87202-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:09.209751 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:09.209316    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8cba866d-6b74-4efa-802d-3d4248a87202-crio-socket\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:09.209751 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:59:09.209396    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 20:59:09.209751 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:59:09.209422    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 20:59:09.209751 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:09.209445    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8cba866d-6b74-4efa-802d-3d4248a87202-crio-socket\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:09.209751 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:59:09.209520    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cba866d-6b74-4efa-802d-3d4248a87202-insights-runtime-extractor-tls podName:8cba866d-6b74-4efa-802d-3d4248a87202 nodeName:}" failed. No retries permitted until 2026-09-15 20:59:09.709477055 +0000 UTC m=+86.251639679 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8cba866d-6b74-4efa-802d-3d4248a87202-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xc4sd" (UID: "8cba866d-6b74-4efa-802d-3d4248a87202") : secret "insights-runtime-extractor-tls" not found
Sep 15 20:59:09.209751 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:59:09.209542    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8cba866d-6b74-4efa-802d-3d4248a87202-kube-rbac-proxy-cm podName:8cba866d-6b74-4efa-802d-3d4248a87202 nodeName:}" failed. No retries permitted until 2026-09-15 20:59:09.709531707 +0000 UTC m=+86.251694337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8cba866d-6b74-4efa-802d-3d4248a87202-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xc4sd" (UID: "8cba866d-6b74-4efa-802d-3d4248a87202") : configmap "kube-rbac-proxy" not found
Sep 15 20:59:09.210092 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:09.210069    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8cba866d-6b74-4efa-802d-3d4248a87202-data-volume\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:09.220867 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:09.220843    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8lnm\" (UniqueName: \"kubernetes.io/projected/8cba866d-6b74-4efa-802d-3d4248a87202-kube-api-access-j8lnm\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:09.714844 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:09.714052    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8cba866d-6b74-4efa-802d-3d4248a87202-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:09.714844 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:09.714114    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cba866d-6b74-4efa-802d-3d4248a87202-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:09.714844 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:59:09.714375    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 20:59:09.714844 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:59:09.714428    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8cba866d-6b74-4efa-802d-3d4248a87202-kube-rbac-proxy-cm podName:8cba866d-6b74-4efa-802d-3d4248a87202 nodeName:}" failed. No retries permitted until 2026-09-15 20:59:10.714414126 +0000 UTC m=+87.256576739 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8cba866d-6b74-4efa-802d-3d4248a87202-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xc4sd" (UID: "8cba866d-6b74-4efa-802d-3d4248a87202") : configmap "kube-rbac-proxy" not found
Sep 15 20:59:09.714844 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:59:09.714613    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 20:59:09.714844 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:59:09.714667    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cba866d-6b74-4efa-802d-3d4248a87202-insights-runtime-extractor-tls podName:8cba866d-6b74-4efa-802d-3d4248a87202 nodeName:}" failed. No retries permitted until 2026-09-15 20:59:10.714650197 +0000 UTC m=+87.256812825 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8cba866d-6b74-4efa-802d-3d4248a87202-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xc4sd" (UID: "8cba866d-6b74-4efa-802d-3d4248a87202") : secret "insights-runtime-extractor-tls" not found
Sep 15 20:59:10.379777 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:10.379743    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rqmrf_04865999-e56e-4a65-9c4a-b52647e7b1db/dns/0.log"
Sep 15 20:59:10.459205 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:10.459168    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:10.511664 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:10.511622    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:10.579435 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:10.579408    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rqmrf_04865999-e56e-4a65-9c4a-b52647e7b1db/kube-rbac-proxy/0.log"
Sep 15 20:59:10.725137 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:10.725041    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8cba866d-6b74-4efa-802d-3d4248a87202-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:10.725137 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:10.725099    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cba866d-6b74-4efa-802d-3d4248a87202-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:10.725379 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:59:10.725179    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 20:59:10.725379 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:59:10.725261    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 20:59:10.725379 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:59:10.725275    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8cba866d-6b74-4efa-802d-3d4248a87202-kube-rbac-proxy-cm podName:8cba866d-6b74-4efa-802d-3d4248a87202 nodeName:}" failed. No retries permitted until 2026-09-15 20:59:12.725253099 +0000 UTC m=+89.267415734 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8cba866d-6b74-4efa-802d-3d4248a87202-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xc4sd" (UID: "8cba866d-6b74-4efa-802d-3d4248a87202") : configmap "kube-rbac-proxy" not found
Sep 15 20:59:10.725379 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:59:10.725336    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cba866d-6b74-4efa-802d-3d4248a87202-insights-runtime-extractor-tls podName:8cba866d-6b74-4efa-802d-3d4248a87202 nodeName:}" failed. No retries permitted until 2026-09-15 20:59:12.725316873 +0000 UTC m=+89.267479500 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8cba866d-6b74-4efa-802d-3d4248a87202-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xc4sd" (UID: "8cba866d-6b74-4efa-802d-3d4248a87202") : secret "insights-runtime-extractor-tls" not found
Sep 15 20:59:10.780510 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:10.780469    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mt7b6_7c4d34d1-b3a5-4974-9db9-0ccf8786db47/dns-node-resolver/0.log"
Sep 15 20:59:11.379144 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:11.379107    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-57cf586d6b-mkkv4_dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e/registry/0.log"
Sep 15 20:59:11.579967 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:11.579933    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-846b7dcc97-fpgwt_2243bd69-2640-4dc6-beaa-7e22b414db60/registry/0.log"
Sep 15 20:59:12.082987 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:12.082943    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:12.180176 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:12.180150    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cdrtr_afadf0fd-9154-4a32-8abe-d8b29dfcaa90/node-ca/0.log"
Sep 15 20:59:12.182658 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:12.182623    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:12.454397 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:12.454354    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:12.503859 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:12.503824    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:12.580118 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:12.580088    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-ddcc97c8b-72s7f_34f22718-53a1-41fd-8eda-f8581983bf3d/router/0.log"
Sep 15 20:59:12.694701 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:12.694645    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:12.745666 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:12.745569    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8cba866d-6b74-4efa-802d-3d4248a87202-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:12.745666 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:12.745621    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cba866d-6b74-4efa-802d-3d4248a87202-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:12.745906 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:59:12.745722    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 20:59:12.745906 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:59:12.745772    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 20:59:12.745906 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:59:12.745796    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8cba866d-6b74-4efa-802d-3d4248a87202-kube-rbac-proxy-cm podName:8cba866d-6b74-4efa-802d-3d4248a87202 nodeName:}" failed. No retries permitted until 2026-09-15 20:59:16.74578062 +0000 UTC m=+93.287943233 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8cba866d-6b74-4efa-802d-3d4248a87202-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xc4sd" (UID: "8cba866d-6b74-4efa-802d-3d4248a87202") : configmap "kube-rbac-proxy" not found
Sep 15 20:59:12.745906 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:59:12.745828    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cba866d-6b74-4efa-802d-3d4248a87202-insights-runtime-extractor-tls podName:8cba866d-6b74-4efa-802d-3d4248a87202 nodeName:}" failed. No retries permitted until 2026-09-15 20:59:16.745811445 +0000 UTC m=+93.287974073 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8cba866d-6b74-4efa-802d-3d4248a87202-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xc4sd" (UID: "8cba866d-6b74-4efa-802d-3d4248a87202") : secret "insights-runtime-extractor-tls" not found
Sep 15 20:59:12.778833 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:12.778799    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5mvsw_2aae8637-fdbb-4c3c-a878-ec6ea9a25c7b/serve-healthcheck-canary/0.log"
Sep 15 20:59:12.794205 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:12.794173    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:12.905866 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:12.905824    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:13.007299 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:13.007209    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:13.062687 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:13.062651    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:15.176713 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:15.176674    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:15.794081 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:15.794037    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:15.894392 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:15.894289    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:16.276786 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:16.276744    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:16.789077 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:16.789043    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8cba866d-6b74-4efa-802d-3d4248a87202-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:16.789240 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:16.789093    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cba866d-6b74-4efa-802d-3d4248a87202-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:16.789240 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:59:16.789192    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 20:59:16.789336 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:59:16.789274    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8cba866d-6b74-4efa-802d-3d4248a87202-kube-rbac-proxy-cm podName:8cba866d-6b74-4efa-802d-3d4248a87202 nodeName:}" failed. No retries permitted until 2026-09-15 20:59:24.78925158 +0000 UTC m=+101.331414215 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8cba866d-6b74-4efa-802d-3d4248a87202-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xc4sd" (UID: "8cba866d-6b74-4efa-802d-3d4248a87202") : configmap "kube-rbac-proxy" not found
Sep 15 20:59:16.789336 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:59:16.789324    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 20:59:16.789423 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:59:16.789406    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cba866d-6b74-4efa-802d-3d4248a87202-insights-runtime-extractor-tls podName:8cba866d-6b74-4efa-802d-3d4248a87202 nodeName:}" failed. No retries permitted until 2026-09-15 20:59:24.789385921 +0000 UTC m=+101.331548539 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8cba866d-6b74-4efa-802d-3d4248a87202-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xc4sd" (UID: "8cba866d-6b74-4efa-802d-3d4248a87202") : secret "insights-runtime-extractor-tls" not found
Sep 15 20:59:16.835804 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:16.835692    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:17.051607 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.051515    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:17.105802 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.105761    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6pkbm"]
Sep 15 20:59:17.114108 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.114082    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.116769 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.116692    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 15 20:59:17.117066 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.117049    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 15 20:59:17.117623 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.117601    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 15 20:59:17.118076 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.118058    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pcj6b\""
Sep 15 20:59:17.118141 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.118091    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 15 20:59:17.142205 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.142081    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:17.193777 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.193627    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c59fe925-61ba-436a-b3e0-3de370d5d0e3-node-exporter-accelerators-collector-config\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.193777 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.193685    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c59fe925-61ba-436a-b3e0-3de370d5d0e3-node-exporter-wtmp\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.193777 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.193721    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c59fe925-61ba-436a-b3e0-3de370d5d0e3-root\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.194064 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.193820    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c59fe925-61ba-436a-b3e0-3de370d5d0e3-sys\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.194064 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.193882    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c59fe925-61ba-436a-b3e0-3de370d5d0e3-node-exporter-textfile\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.194064 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.193907    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw69m\" (UniqueName: \"kubernetes.io/projected/c59fe925-61ba-436a-b3e0-3de370d5d0e3-kube-api-access-jw69m\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.194064 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.193948    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c59fe925-61ba-436a-b3e0-3de370d5d0e3-node-exporter-tls\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.194064 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.193988    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c59fe925-61ba-436a-b3e0-3de370d5d0e3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.194064 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.194020    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c59fe925-61ba-436a-b3e0-3de370d5d0e3-metrics-client-ca\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.203941 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.203904    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:17.294808 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.294770    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c59fe925-61ba-436a-b3e0-3de370d5d0e3-sys\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.295281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.294854    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c59fe925-61ba-436a-b3e0-3de370d5d0e3-node-exporter-textfile\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.295281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.294881    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw69m\" (UniqueName: \"kubernetes.io/projected/c59fe925-61ba-436a-b3e0-3de370d5d0e3-kube-api-access-jw69m\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.295281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.294927    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c59fe925-61ba-436a-b3e0-3de370d5d0e3-node-exporter-tls\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.295281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.294962    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c59fe925-61ba-436a-b3e0-3de370d5d0e3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.295281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.294990    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c59fe925-61ba-436a-b3e0-3de370d5d0e3-metrics-client-ca\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.295281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.295133    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c59fe925-61ba-436a-b3e0-3de370d5d0e3-node-exporter-accelerators-collector-config\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.295281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.295179    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c59fe925-61ba-436a-b3e0-3de370d5d0e3-node-exporter-wtmp\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.295281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.295211    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c59fe925-61ba-436a-b3e0-3de370d5d0e3-root\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.295736 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.295350    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c59fe925-61ba-436a-b3e0-3de370d5d0e3-root\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.295736 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.295673    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c59fe925-61ba-436a-b3e0-3de370d5d0e3-metrics-client-ca\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.295843 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.295739    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c59fe925-61ba-436a-b3e0-3de370d5d0e3-sys\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.295933 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.295912    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c59fe925-61ba-436a-b3e0-3de370d5d0e3-node-exporter-accelerators-collector-config\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.295993 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.295952    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c59fe925-61ba-436a-b3e0-3de370d5d0e3-node-exporter-textfile\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.296070 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.296054    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c59fe925-61ba-436a-b3e0-3de370d5d0e3-node-exporter-wtmp\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.299286 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.299256    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c59fe925-61ba-436a-b3e0-3de370d5d0e3-node-exporter-tls\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.300422 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.300398    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c59fe925-61ba-436a-b3e0-3de370d5d0e3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.307070 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.307004    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw69m\" (UniqueName: \"kubernetes.io/projected/c59fe925-61ba-436a-b3e0-3de370d5d0e3-kube-api-access-jw69m\") pod \"node-exporter-6pkbm\" (UID: \"c59fe925-61ba-436a-b3e0-3de370d5d0e3\") " pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.308121 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.308080    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:17.403621 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.403586    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:17.426275 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.426234    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6pkbm"
Sep 15 20:59:17.504758 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.504725    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:17.631567 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.631531    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:17.942577 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:17.942459    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:18.103403 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:18.103181    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:18.166836 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:18.166807    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:59:18.176571 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:18.176537    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:18.296215 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:18.296131    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:18.521826 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:18.521781    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:19.982849 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:59:19.982176    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc59fe925_61ba_436a_b3e0_3de370d5d0e3.slice/crio-60e0a827fc6f44f252094056812f92085c8a5ba2bd7c2095be4662332948efeb WatchSource:0}: Error finding container 60e0a827fc6f44f252094056812f92085c8a5ba2bd7c2095be4662332948efeb: Status 404 returned error can't find the container with id 60e0a827fc6f44f252094056812f92085c8a5ba2bd7c2095be4662332948efeb
Sep 15 20:59:20.021388 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:20.021355    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:20.121715 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:20.121670    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:20.340246 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:20.340221    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-57cf586d6b-mkkv4"]
Sep 15 20:59:20.346457 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:20.346429    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:59:20.364898 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:20.364866    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:20.445133 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:20.445100    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:20.573543 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:20.573473    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:20.592157 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:20.592074    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-n5nqb" event={"ID":"5c896e08-2021-4261-8701-ffb73d0bbed9","Type":"ContainerStarted","Data":"40f1ca1f5b9b492aa16adb85293414789928bda15674f16158e0a5f477577bdd"}
Sep 15 20:59:20.592473 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:20.592449    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-n5nqb"
Sep 15 20:59:20.593663 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:20.593634    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6pkbm" event={"ID":"c59fe925-61ba-436a-b3e0-3de370d5d0e3","Type":"ContainerStarted","Data":"60e0a827fc6f44f252094056812f92085c8a5ba2bd7c2095be4662332948efeb"}
Sep 15 20:59:20.612119 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:20.612063    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-n5nqb" podStartSLOduration=1.14905854 podStartE2EDuration="22.612043987s" totalImagesPullingTime="21.462985447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 20:58:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:20.611119624 +0000 UTC m=+97.153282286" watchObservedRunningTime="2026-09-15 20:59:20.612043987 +0000 UTC m=+97.154206667"
Sep 15 20:59:20.619750 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:20.619722    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-n5nqb"
Sep 15 20:59:20.640169 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:20.640134    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:20.693970 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:20.693927    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:21.122920 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:21.122855    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:21.179914 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:21.179879    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:21.573456 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:21.573366    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:21.673272 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:21.673233    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:21.774606 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:21.774559    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:21.877251 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:21.877212    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:21.974575 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:21.974533    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:22.073668 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:22.073627    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:22.481240 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:22.481200    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:22.612294 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:22.612250    2564 generic.go:356] "Generic (PLEG): container finished" podID="c59fe925-61ba-436a-b3e0-3de370d5d0e3" containerID="14ca656ac3ae05f370f0738799e45649bf44ff67954dd464d09abaf13c7d4ad3" exitCode=0
Sep 15 20:59:22.612522 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:22.612342    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6pkbm" event={"ID":"c59fe925-61ba-436a-b3e0-3de370d5d0e3","Type":"ContainerDied","Data":"14ca656ac3ae05f370f0738799e45649bf44ff67954dd464d09abaf13c7d4ad3"}
Sep 15 20:59:23.183284 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.183189    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt" podUID="2243bd69-2640-4dc6-beaa-7e22b414db60" containerName="registry" containerID="cri-o://3de31086882d5b0c696c11f710e50cf3fc6b9a5e6aaa686927b28715a5b76703" gracePeriod=30
Sep 15 20:59:23.347924 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.347844    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:23.500599 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.500568    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:59:23.564918 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.564880    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2243bd69-2640-4dc6-beaa-7e22b414db60-installation-pull-secrets\") pod \"2243bd69-2640-4dc6-beaa-7e22b414db60\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") "
Sep 15 20:59:23.565083 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.564978    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-certificates\") pod \"2243bd69-2640-4dc6-beaa-7e22b414db60\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") "
Sep 15 20:59:23.565083 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.565020    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls\") pod \"2243bd69-2640-4dc6-beaa-7e22b414db60\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") "
Sep 15 20:59:23.565083 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.565062    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2243bd69-2640-4dc6-beaa-7e22b414db60-trusted-ca\") pod \"2243bd69-2640-4dc6-beaa-7e22b414db60\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") "
Sep 15 20:59:23.565227 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.565118    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2243bd69-2640-4dc6-beaa-7e22b414db60-image-registry-private-configuration\") pod \"2243bd69-2640-4dc6-beaa-7e22b414db60\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") "
Sep 15 20:59:23.565227 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.565166    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72wcj\" (UniqueName: \"kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-kube-api-access-72wcj\") pod \"2243bd69-2640-4dc6-beaa-7e22b414db60\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") "
Sep 15 20:59:23.565227 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.565195    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2243bd69-2640-4dc6-beaa-7e22b414db60-ca-trust-extracted\") pod \"2243bd69-2640-4dc6-beaa-7e22b414db60\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") "
Sep 15 20:59:23.565227 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.565225    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-bound-sa-token\") pod \"2243bd69-2640-4dc6-beaa-7e22b414db60\" (UID: \"2243bd69-2640-4dc6-beaa-7e22b414db60\") "
Sep 15 20:59:23.565448 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.565393    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2243bd69-2640-4dc6-beaa-7e22b414db60" (UID: "2243bd69-2640-4dc6-beaa-7e22b414db60"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:59:23.565604 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.565580    2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-certificates\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 20:59:23.565929 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.565902    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2243bd69-2640-4dc6-beaa-7e22b414db60-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2243bd69-2640-4dc6-beaa-7e22b414db60" (UID: "2243bd69-2640-4dc6-beaa-7e22b414db60"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:59:23.569392 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.569344    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2243bd69-2640-4dc6-beaa-7e22b414db60" (UID: "2243bd69-2640-4dc6-beaa-7e22b414db60"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:59:23.569546 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.569426    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2243bd69-2640-4dc6-beaa-7e22b414db60" (UID: "2243bd69-2640-4dc6-beaa-7e22b414db60"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:59:23.569616 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.569312    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2243bd69-2640-4dc6-beaa-7e22b414db60-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2243bd69-2640-4dc6-beaa-7e22b414db60" (UID: "2243bd69-2640-4dc6-beaa-7e22b414db60"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:59:23.569750 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.569724    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2243bd69-2640-4dc6-beaa-7e22b414db60-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2243bd69-2640-4dc6-beaa-7e22b414db60" (UID: "2243bd69-2640-4dc6-beaa-7e22b414db60"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:59:23.569827 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.569786    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-kube-api-access-72wcj" (OuterVolumeSpecName: "kube-api-access-72wcj") pod "2243bd69-2640-4dc6-beaa-7e22b414db60" (UID: "2243bd69-2640-4dc6-beaa-7e22b414db60"). InnerVolumeSpecName "kube-api-access-72wcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:59:23.576410 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.576327    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2243bd69-2640-4dc6-beaa-7e22b414db60-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2243bd69-2640-4dc6-beaa-7e22b414db60" (UID: "2243bd69-2640-4dc6-beaa-7e22b414db60"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:59:23.619680 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.619637    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6pkbm" event={"ID":"c59fe925-61ba-436a-b3e0-3de370d5d0e3","Type":"ContainerStarted","Data":"9ec0328249cd680226c9814b1bbaaf4fd1b724219591b17934c2fe4c00f4106b"}
Sep 15 20:59:23.619845 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.619712    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6pkbm" event={"ID":"c59fe925-61ba-436a-b3e0-3de370d5d0e3","Type":"ContainerStarted","Data":"28676d80aaf7e1584a77f298b99a745e5822b147a4426e6a8db096b594672755"}
Sep 15 20:59:23.621334 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.621305    2564 generic.go:356] "Generic (PLEG): container finished" podID="2243bd69-2640-4dc6-beaa-7e22b414db60" containerID="3de31086882d5b0c696c11f710e50cf3fc6b9a5e6aaa686927b28715a5b76703" exitCode=0
Sep 15 20:59:23.621443 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.621363    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt"
Sep 15 20:59:23.621443 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.621393    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt" event={"ID":"2243bd69-2640-4dc6-beaa-7e22b414db60","Type":"ContainerDied","Data":"3de31086882d5b0c696c11f710e50cf3fc6b9a5e6aaa686927b28715a5b76703"}
Sep 15 20:59:23.621551 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.621446    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-846b7dcc97-fpgwt" event={"ID":"2243bd69-2640-4dc6-beaa-7e22b414db60","Type":"ContainerDied","Data":"075c7e8868b48fd5c3cbfce2506001c379ff691074bd4ecb3618c59766f53445"}
Sep 15 20:59:23.621551 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.621472    2564 scope.go:117] "RemoveContainer" containerID="3de31086882d5b0c696c11f710e50cf3fc6b9a5e6aaa686927b28715a5b76703"
Sep 15 20:59:23.632332 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.632310    2564 scope.go:117] "RemoveContainer" containerID="3de31086882d5b0c696c11f710e50cf3fc6b9a5e6aaa686927b28715a5b76703"
Sep 15 20:59:23.633045 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:59:23.633010    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3de31086882d5b0c696c11f710e50cf3fc6b9a5e6aaa686927b28715a5b76703\": container with ID starting with 3de31086882d5b0c696c11f710e50cf3fc6b9a5e6aaa686927b28715a5b76703 not found: ID does not exist" containerID="3de31086882d5b0c696c11f710e50cf3fc6b9a5e6aaa686927b28715a5b76703"
Sep 15 20:59:23.633139 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.633058    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3de31086882d5b0c696c11f710e50cf3fc6b9a5e6aaa686927b28715a5b76703"} err="failed to get container status \"3de31086882d5b0c696c11f710e50cf3fc6b9a5e6aaa686927b28715a5b76703\": rpc error: code = NotFound desc = could not find container \"3de31086882d5b0c696c11f710e50cf3fc6b9a5e6aaa686927b28715a5b76703\": container with ID starting with 3de31086882d5b0c696c11f710e50cf3fc6b9a5e6aaa686927b28715a5b76703 not found: ID does not exist"
Sep 15 20:59:23.645913 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.645843    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6pkbm" podStartSLOduration=4.548392346 podStartE2EDuration="6.645820292s" totalImagesPullingTime="1.972026049s" totalInitContainerRuntime="125.401897ms" isStatefulPod=true podCreationTimestamp="2026-09-15 20:59:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:23.635872615 +0000 UTC m=+100.178035287" watchObservedRunningTime="2026-09-15 20:59:23.645820292 +0000 UTC m=+100.187982928"
Sep 15 20:59:23.653551 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.653515    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-846b7dcc97-fpgwt"]
Sep 15 20:59:23.657070 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.657042    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-846b7dcc97-fpgwt"]
Sep 15 20:59:23.666542 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.666481    2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-registry-tls\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 20:59:23.666542 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.666544    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2243bd69-2640-4dc6-beaa-7e22b414db60-trusted-ca\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 20:59:23.666809 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.666560    2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2243bd69-2640-4dc6-beaa-7e22b414db60-image-registry-private-configuration\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 20:59:23.666809 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.666576    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72wcj\" (UniqueName: \"kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-kube-api-access-72wcj\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 20:59:23.666809 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.666592    2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2243bd69-2640-4dc6-beaa-7e22b414db60-ca-trust-extracted\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 20:59:23.666809 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.666605    2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2243bd69-2640-4dc6-beaa-7e22b414db60-bound-sa-token\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 20:59:23.666809 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.666617    2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2243bd69-2640-4dc6-beaa-7e22b414db60-installation-pull-secrets\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 20:59:23.973282 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:23.973247    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2243bd69-2640-4dc6-beaa-7e22b414db60" path="/var/lib/kubelet/pods/2243bd69-2640-4dc6-beaa-7e22b414db60/volumes"
Sep 15 20:59:24.867613 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:24.867576    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:24.876659 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:24.876627    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8cba866d-6b74-4efa-802d-3d4248a87202-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:24.876823 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:24.876669    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cba866d-6b74-4efa-802d-3d4248a87202-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:24.877321 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:24.877296    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8cba866d-6b74-4efa-802d-3d4248a87202-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:24.884844 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:24.884816    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8cba866d-6b74-4efa-802d-3d4248a87202-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xc4sd\" (UID: \"8cba866d-6b74-4efa-802d-3d4248a87202\") " pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:24.968028 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:24.967996    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:25.129460 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:25.129374    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xc4sd"
Sep 15 20:59:25.433464 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:25.433392    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xc4sd"]
Sep 15 20:59:25.437596 ip-10-0-138-221 kubenswrapper[2564]: W0915 20:59:25.437561    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cba866d_6b74_4efa_802d_3d4248a87202.slice/crio-9e213db2211592d5f56445dc10d54a7228466851843126bacaab16452e79ad54 WatchSource:0}: Error finding container 9e213db2211592d5f56445dc10d54a7228466851843126bacaab16452e79ad54: Status 404 returned error can't find the container with id 9e213db2211592d5f56445dc10d54a7228466851843126bacaab16452e79ad54
Sep 15 20:59:25.634667 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:25.634628    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xc4sd" event={"ID":"8cba866d-6b74-4efa-802d-3d4248a87202","Type":"ContainerStarted","Data":"ba7acf60748c921f65145c5bdb39e0de34bf0f3bbae87a715811ddd02a42957f"}
Sep 15 20:59:25.634841 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:25.634701    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xc4sd" event={"ID":"8cba866d-6b74-4efa-802d-3d4248a87202","Type":"ContainerStarted","Data":"9e213db2211592d5f56445dc10d54a7228466851843126bacaab16452e79ad54"}
Sep 15 20:59:27.646241 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:27.646193    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xc4sd" event={"ID":"8cba866d-6b74-4efa-802d-3d4248a87202","Type":"ContainerStarted","Data":"05b5eff0aeb1f6b0c70350650f1c3e052a404d457f1a1f58b3fadae256a275ba"}
Sep 15 20:59:28.545125 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:28.545046    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:29.660275 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:29.660233    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xc4sd" event={"ID":"8cba866d-6b74-4efa-802d-3d4248a87202","Type":"ContainerStarted","Data":"6bf9300f6a601af41b9fa6d5a4a687037cbb6aa872771d6e6b2ab995ce2c2e42"}
Sep 15 20:59:29.695910 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:29.695839    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xc4sd" podStartSLOduration=18.139617489 podStartE2EDuration="21.695813889s" totalImagesPullingTime="3.5561964s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 20:59:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 20:59:29.693395752 +0000 UTC m=+106.235558408" watchObservedRunningTime="2026-09-15 20:59:29.695813889 +0000 UTC m=+106.237976525"
Sep 15 20:59:30.166339 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:30.166298    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:30.223010 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:30.222969    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:30.584873 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:30.584786    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:30.685451 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:30.685416    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:33.349744 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:33.349712    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:36.900100 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:36.900047    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:36.998032 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:36.998000    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:39.280680 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:39.280641    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:39.378438 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:39.378399    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:45.363704 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.363635    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4" podUID="dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e" containerName="registry" containerID="cri-o://132eae7c64a215fc85da0d0543892f7c75d187907c6adec7bc6232fa2e52708c" gracePeriod=30
Sep 15 20:59:45.627468 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.627385    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:59:45.722820 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.722784    2564 generic.go:356] "Generic (PLEG): container finished" podID="dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e" containerID="132eae7c64a215fc85da0d0543892f7c75d187907c6adec7bc6232fa2e52708c" exitCode=0
Sep 15 20:59:45.722999 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.722831    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4"
Sep 15 20:59:45.722999 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.722866    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4" event={"ID":"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e","Type":"ContainerDied","Data":"132eae7c64a215fc85da0d0543892f7c75d187907c6adec7bc6232fa2e52708c"}
Sep 15 20:59:45.722999 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.722916    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57cf586d6b-mkkv4" event={"ID":"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e","Type":"ContainerDied","Data":"b3f4220f18ca292b941539f607e788ecbd5a1bd4f87c182e1e10f56db087a81c"}
Sep 15 20:59:45.722999 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.722937    2564 scope.go:117] "RemoveContainer" containerID="132eae7c64a215fc85da0d0543892f7c75d187907c6adec7bc6232fa2e52708c"
Sep 15 20:59:45.731810 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.731791    2564 scope.go:117] "RemoveContainer" containerID="132eae7c64a215fc85da0d0543892f7c75d187907c6adec7bc6232fa2e52708c"
Sep 15 20:59:45.732129 ip-10-0-138-221 kubenswrapper[2564]: E0915 20:59:45.732107    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"132eae7c64a215fc85da0d0543892f7c75d187907c6adec7bc6232fa2e52708c\": container with ID starting with 132eae7c64a215fc85da0d0543892f7c75d187907c6adec7bc6232fa2e52708c not found: ID does not exist" containerID="132eae7c64a215fc85da0d0543892f7c75d187907c6adec7bc6232fa2e52708c"
Sep 15 20:59:45.732281 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.732139    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"132eae7c64a215fc85da0d0543892f7c75d187907c6adec7bc6232fa2e52708c"} err="failed to get container status \"132eae7c64a215fc85da0d0543892f7c75d187907c6adec7bc6232fa2e52708c\": rpc error: code = NotFound desc = could not find container \"132eae7c64a215fc85da0d0543892f7c75d187907c6adec7bc6232fa2e52708c\": container with ID starting with 132eae7c64a215fc85da0d0543892f7c75d187907c6adec7bc6232fa2e52708c not found: ID does not exist"
Sep 15 20:59:45.767468 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.767431    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-installation-pull-secrets\") pod \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") "
Sep 15 20:59:45.767468 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.767468    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqq64\" (UniqueName: \"kubernetes.io/projected/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-kube-api-access-nqq64\") pod \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") "
Sep 15 20:59:45.767701 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.767526    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-ca-trust-extracted\") pod \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") "
Sep 15 20:59:45.767701 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.767552    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-trusted-ca\") pod \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") "
Sep 15 20:59:45.767701 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.767572    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-registry-tls\") pod \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") "
Sep 15 20:59:45.767701 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.767610    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-bound-sa-token\") pod \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") "
Sep 15 20:59:45.767701 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.767658    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-registry-certificates\") pod \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") "
Sep 15 20:59:45.767942 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.767703    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-image-registry-private-configuration\") pod \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\" (UID: \"dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e\") "
Sep 15 20:59:45.768424 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.768132    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e" (UID: "dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:59:45.768959 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.768924    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e" (UID: "dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 20:59:45.770143 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.770118    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e" (UID: "dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:59:45.770143 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.770143    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-kube-api-access-nqq64" (OuterVolumeSpecName: "kube-api-access-nqq64") pod "dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e" (UID: "dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e"). InnerVolumeSpecName "kube-api-access-nqq64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:59:45.770371 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.770353    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e" (UID: "dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 20:59:45.770567 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.770540    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e" (UID: "dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:59:45.770656 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.770637    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e" (UID: "dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 20:59:45.776121 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.776091    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e" (UID: "dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 20:59:45.869310 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.869268    2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-bound-sa-token\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 20:59:45.869310 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.869302    2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-registry-certificates\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 20:59:45.869310 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.869313    2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-image-registry-private-configuration\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 20:59:45.869561 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.869324    2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-installation-pull-secrets\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 20:59:45.869561 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.869333    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqq64\" (UniqueName: \"kubernetes.io/projected/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-kube-api-access-nqq64\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 20:59:45.869561 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.869341    2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-ca-trust-extracted\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 20:59:45.869561 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.869351    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-trusted-ca\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 20:59:45.869561 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:45.869359    2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e-registry-tls\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 20:59:46.040651 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:46.040615    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-57cf586d6b-mkkv4"]
Sep 15 20:59:46.044092 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:46.044068    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-57cf586d6b-mkkv4"]
Sep 15 20:59:46.892947 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:46.892912    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:47.971664 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:47.971634    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e" path="/var/lib/kubelet/pods/dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e/volumes"
Sep 15 20:59:51.896599 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:51.896568    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:51.993373 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:51.993340    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:58.941819 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:58.941780    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:59.043038 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:59.043000    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 20:59:59.096104 ip-10-0-138-221 kubenswrapper[2564]: I0915 20:59:59.096072    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:00:00.783872 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:00:00.783836    2564 generic.go:356] "Generic (PLEG): container finished" podID="f3b99f56-873f-40d8-abf3-b0c02523b2f7" containerID="4633ea3e4451167ea2ab658f86e7085ee63ac9c2de4b96a350acbb03d81044d2" exitCode=0
Sep 15 21:00:00.784293 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:00:00.783911    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ppbrv" event={"ID":"f3b99f56-873f-40d8-abf3-b0c02523b2f7","Type":"ContainerDied","Data":"4633ea3e4451167ea2ab658f86e7085ee63ac9c2de4b96a350acbb03d81044d2"}
Sep 15 21:00:00.784293 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:00:00.784273    2564 scope.go:117] "RemoveContainer" containerID="4633ea3e4451167ea2ab658f86e7085ee63ac9c2de4b96a350acbb03d81044d2"
Sep 15 21:00:00.785593 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:00:00.785571    2564 generic.go:356] "Generic (PLEG): container finished" podID="dcfdcceb-cea6-4f5a-958f-a1b558bc1a1f" containerID="1ee60dcf38da5b9169bf5255c39ef82e4323b3d3259d6aac8659cf86a6232558" exitCode=0
Sep 15 21:00:00.785690 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:00:00.785601    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jsn2f" event={"ID":"dcfdcceb-cea6-4f5a-958f-a1b558bc1a1f","Type":"ContainerDied","Data":"1ee60dcf38da5b9169bf5255c39ef82e4323b3d3259d6aac8659cf86a6232558"}
Sep 15 21:00:00.785847 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:00:00.785834    2564 scope.go:117] "RemoveContainer" containerID="1ee60dcf38da5b9169bf5255c39ef82e4323b3d3259d6aac8659cf86a6232558"
Sep 15 21:00:01.791442 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:00:01.791396    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-jsn2f" event={"ID":"dcfdcceb-cea6-4f5a-958f-a1b558bc1a1f","Type":"ContainerStarted","Data":"d44aa9c8c41954d3ad6fe124c06cc638ec144b18719a836e8aaa6dc3b1d573ef"}
Sep 15 21:00:01.793383 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:00:01.793357    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ppbrv" event={"ID":"f3b99f56-873f-40d8-abf3-b0c02523b2f7","Type":"ContainerStarted","Data":"27b4903e62215c28b5c8257d8b9de0a93ab5a2e18914cc3b59700c9b4c613eed"}
Sep 15 21:00:33.174004 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:00:33.173965    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:00:33.229008 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:00:33.228963    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:00:39.175464 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:00:39.175372    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:00:39.235365 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:00:39.235333    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:00:45.150017 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:00:45.149973    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:00:45.224250 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:00:45.224211    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:00:51.907397 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:00:51.907362    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:00:52.004432 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:00:52.004395    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:00:53.438637 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:00:53.438593    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:00:53.494357 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:00:53.494318    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:00:53.644324 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:00:53.644289    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:00:53.708920 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:00:53.708837    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:00:54.216620 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:00:54.216582    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:00:54.297410 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:00:54.297375    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:00:55.094306 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:00:55.094265    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:01:06.287997 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:01:06.287961    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:01:06.377006 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:01:06.376974    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:01:18.620100 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:01:18.620065    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:01:18.749865 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:01:18.749827    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:01:46.078610 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:01:46.078574    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:01:47.456596 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:01:47.456560    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:01:49.690575 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:01:49.690536    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:01:49.797723 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:01:49.797688    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:01:51.328320 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:01:51.328283    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:01:51.480338 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:01:51.480305    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:01:51.604950 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:01:51.604914    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:01:53.983833 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:01:53.983794    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:01:54.198874 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:01:54.198836    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:01:56.500247 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:01:56.500212    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:01:56.726324 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:01:56.726284    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:00.941050 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:00.941011    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:01.037095 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:01.037056    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:01.203976 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:01.203887    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:08.115131 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:08.114827    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:10.341748 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:10.341708    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:13.883986 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:13.883939    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:13.983777 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:13.983732    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:15.729522 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:15.729464    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:25.551109 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:25.551066    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:30.407986 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:30.407942    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:30.512301 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:30.512258    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:31.117876 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.117841    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:31.327110 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.327071    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:31.439795 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.439696    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:31.479207 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.479173    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6"]
Sep 15 21:02:31.479535 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.479522    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e" containerName="registry"
Sep 15 21:02:31.479601 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.479538    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e" containerName="registry"
Sep 15 21:02:31.479601 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.479560    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2243bd69-2640-4dc6-beaa-7e22b414db60" containerName="registry"
Sep 15 21:02:31.479601 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.479568    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2243bd69-2640-4dc6-beaa-7e22b414db60" containerName="registry"
Sep 15 21:02:31.479691 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.479647    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbbd33a2-7ea1-4344-ab0e-ef5e6494bc6e" containerName="registry"
Sep 15 21:02:31.479691 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.479656    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2243bd69-2640-4dc6-beaa-7e22b414db60" containerName="registry"
Sep 15 21:02:31.481873 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.481855    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6"
Sep 15 21:02:31.485959 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.485925    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 15 21:02:31.485959 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.485955    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 15 21:02:31.486202 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.486187    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 15 21:02:31.486262 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.486243    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 15 21:02:31.486579 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.486561    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-wj45x\""
Sep 15 21:02:31.497734 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.497706    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6"]
Sep 15 21:02:31.585482 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.585439    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f105a3b5-d8f2-42cf-ad9e-b82170d48d88-tmp\") pod \"opendatahub-operator-controller-manager-6f85789df9-7zsl6\" (UID: \"f105a3b5-d8f2-42cf-ad9e-b82170d48d88\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6"
Sep 15 21:02:31.585687 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.585520    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f105a3b5-d8f2-42cf-ad9e-b82170d48d88-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f85789df9-7zsl6\" (UID: \"f105a3b5-d8f2-42cf-ad9e-b82170d48d88\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6"
Sep 15 21:02:31.585687 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.585551    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qglbb\" (UniqueName: \"kubernetes.io/projected/f105a3b5-d8f2-42cf-ad9e-b82170d48d88-kube-api-access-qglbb\") pod \"opendatahub-operator-controller-manager-6f85789df9-7zsl6\" (UID: \"f105a3b5-d8f2-42cf-ad9e-b82170d48d88\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6"
Sep 15 21:02:31.585687 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.585585    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f105a3b5-d8f2-42cf-ad9e-b82170d48d88-manifests\") pod \"opendatahub-operator-controller-manager-6f85789df9-7zsl6\" (UID: \"f105a3b5-d8f2-42cf-ad9e-b82170d48d88\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6"
Sep 15 21:02:31.585687 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.585665    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f105a3b5-d8f2-42cf-ad9e-b82170d48d88-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f85789df9-7zsl6\" (UID: \"f105a3b5-d8f2-42cf-ad9e-b82170d48d88\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6"
Sep 15 21:02:31.607215 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.607183    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:31.686870 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.686836    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f105a3b5-d8f2-42cf-ad9e-b82170d48d88-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f85789df9-7zsl6\" (UID: \"f105a3b5-d8f2-42cf-ad9e-b82170d48d88\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6"
Sep 15 21:02:31.687036 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.686877    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qglbb\" (UniqueName: \"kubernetes.io/projected/f105a3b5-d8f2-42cf-ad9e-b82170d48d88-kube-api-access-qglbb\") pod \"opendatahub-operator-controller-manager-6f85789df9-7zsl6\" (UID: \"f105a3b5-d8f2-42cf-ad9e-b82170d48d88\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6"
Sep 15 21:02:31.687036 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.686914    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f105a3b5-d8f2-42cf-ad9e-b82170d48d88-manifests\") pod \"opendatahub-operator-controller-manager-6f85789df9-7zsl6\" (UID: \"f105a3b5-d8f2-42cf-ad9e-b82170d48d88\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6"
Sep 15 21:02:31.687036 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.686956    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f105a3b5-d8f2-42cf-ad9e-b82170d48d88-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f85789df9-7zsl6\" (UID: \"f105a3b5-d8f2-42cf-ad9e-b82170d48d88\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6"
Sep 15 21:02:31.687036 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.687008    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f105a3b5-d8f2-42cf-ad9e-b82170d48d88-tmp\") pod \"opendatahub-operator-controller-manager-6f85789df9-7zsl6\" (UID: \"f105a3b5-d8f2-42cf-ad9e-b82170d48d88\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6"
Sep 15 21:02:31.687366 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.687346    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f105a3b5-d8f2-42cf-ad9e-b82170d48d88-manifests\") pod \"opendatahub-operator-controller-manager-6f85789df9-7zsl6\" (UID: \"f105a3b5-d8f2-42cf-ad9e-b82170d48d88\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6"
Sep 15 21:02:31.689468 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.689446    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f105a3b5-d8f2-42cf-ad9e-b82170d48d88-tmp\") pod \"opendatahub-operator-controller-manager-6f85789df9-7zsl6\" (UID: \"f105a3b5-d8f2-42cf-ad9e-b82170d48d88\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6"
Sep 15 21:02:31.689627 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.689610    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f105a3b5-d8f2-42cf-ad9e-b82170d48d88-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f85789df9-7zsl6\" (UID: \"f105a3b5-d8f2-42cf-ad9e-b82170d48d88\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6"
Sep 15 21:02:31.689690 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.689672    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f105a3b5-d8f2-42cf-ad9e-b82170d48d88-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f85789df9-7zsl6\" (UID: \"f105a3b5-d8f2-42cf-ad9e-b82170d48d88\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6"
Sep 15 21:02:31.696306 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.696236    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qglbb\" (UniqueName: \"kubernetes.io/projected/f105a3b5-d8f2-42cf-ad9e-b82170d48d88-kube-api-access-qglbb\") pod \"opendatahub-operator-controller-manager-6f85789df9-7zsl6\" (UID: \"f105a3b5-d8f2-42cf-ad9e-b82170d48d88\") " pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6"
Sep 15 21:02:31.704270 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.704243    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:31.792558 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.792521    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6"
Sep 15 21:02:31.805090 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.805056    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:31.918344 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:31.918313    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:32.011996 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:32.011900    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:32.077994 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:32.077963    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6"]
Sep 15 21:02:32.080482 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:02:32.080449    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf105a3b5_d8f2_42cf_ad9e_b82170d48d88.slice/crio-b03497455f94b6af84e84fd8479e13967622843901476675e3890cbe04dee703 WatchSource:0}: Error finding container b03497455f94b6af84e84fd8479e13967622843901476675e3890cbe04dee703: Status 404 returned error can't find the container with id b03497455f94b6af84e84fd8479e13967622843901476675e3890cbe04dee703
Sep 15 21:02:32.376342 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:32.376312    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6" event={"ID":"f105a3b5-d8f2-42cf-ad9e-b82170d48d88","Type":"ContainerStarted","Data":"b03497455f94b6af84e84fd8479e13967622843901476675e3890cbe04dee703"}
Sep 15 21:02:36.398583 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:36.398549    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6" event={"ID":"f105a3b5-d8f2-42cf-ad9e-b82170d48d88","Type":"ContainerStarted","Data":"5ff78b50403b765194f4bec68ff22af11fb6ced874d773a4b6ab7f510c7ec411"}
Sep 15 21:02:37.405009 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:37.404965    2564 generic.go:356] "Generic (PLEG): container finished" podID="f105a3b5-d8f2-42cf-ad9e-b82170d48d88" containerID="5ff78b50403b765194f4bec68ff22af11fb6ced874d773a4b6ab7f510c7ec411" exitCode=0
Sep 15 21:02:37.405650 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:37.405048    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6" event={"ID":"f105a3b5-d8f2-42cf-ad9e-b82170d48d88","Type":"ContainerDied","Data":"5ff78b50403b765194f4bec68ff22af11fb6ced874d773a4b6ab7f510c7ec411"}
Sep 15 21:02:37.937282 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:37.937251    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:38.038661 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:38.038626    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:38.411205 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:38.411171    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6" event={"ID":"f105a3b5-d8f2-42cf-ad9e-b82170d48d88","Type":"ContainerStarted","Data":"78552b63d92c6ba971402f7b5399907d60df98c17274591a5e54ca9a5bbe9a27"}
Sep 15 21:02:38.411618 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:38.411311    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6"
Sep 15 21:02:38.439384 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:38.439321    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6" podStartSLOduration=2.632320542 podStartE2EDuration="7.439305048s" totalImagesPullingTime="3.783907662s" totalInitContainerRuntime="1.023076844s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:02:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:02:38.43705189 +0000 UTC m=+294.979214546" watchObservedRunningTime="2026-09-15 21:02:38.439305048 +0000 UTC m=+294.981467739"
Sep 15 21:02:43.875071 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:43.875026    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-7t46m_c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf/console-operator/1.log"
Sep 15 21:02:43.878015 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:43.877987    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-7t46m_c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf/console-operator/1.log"
Sep 15 21:02:43.879088 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:43.879067    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4c7k_fc205950-caf0-4c47-84ce-a8fc54a1afb2/ovn-acl-logging/0.log"
Sep 15 21:02:43.885377 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:43.885356    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4c7k_fc205950-caf0-4c47-84ce-a8fc54a1afb2/ovn-acl-logging/0.log"
Sep 15 21:02:43.890624 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:43.890604    2564 kubelet.go:1648] "Image garbage collection succeeded"
Sep 15 21:02:45.605781 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:45.605741    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:45.670277 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:45.670239    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:49.417276 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:49.417248    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6f85789df9-7zsl6"
Sep 15 21:02:50.978103 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:50.978065    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:51.098688 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:51.098655    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:51.324683 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:51.324568    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:51.686123 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:51.686088    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:52.902870 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:52.902821    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:53.000418 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:53.000375    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:53.820937 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:53.820891    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:58.392352 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:58.392318    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:58.491324 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:58.491284    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:58.870526 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:58.870466    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:59.044724 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:59.044689    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:02:59.144076 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:02:59.143982    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:03:02.221087 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:02.221050    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:03:02.322023 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:02.321994    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:03:02.422331 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:02.422296    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:03:02.523748 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:02.523659    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:03:02.796614 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:02.796525    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:03:03.093597 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:03.093514    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:03:10.629134 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:10.629096    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5665dbb77c-zzs2w"]
Sep 15 21:03:10.632434 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:10.632415    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5665dbb77c-zzs2w"
Sep 15 21:03:10.635270 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:10.635246    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 15 21:03:10.636456 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:10.636433    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 15 21:03:10.636585 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:10.636436    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 15 21:03:10.636647 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:10.636583    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 15 21:03:10.636647 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:10.636596    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 15 21:03:10.636647 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:10.636625    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-qb8hx\""
Sep 15 21:03:10.641631 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:10.641609    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5665dbb77c-zzs2w"]
Sep 15 21:03:10.716083 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:10.716042    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwwht\" (UniqueName: \"kubernetes.io/projected/8bc5b527-c655-4c33-a3b0-cf77e0660ac3-kube-api-access-fwwht\") pod \"lws-controller-manager-5665dbb77c-zzs2w\" (UID: \"8bc5b527-c655-4c33-a3b0-cf77e0660ac3\") " pod="openshift-lws-operator/lws-controller-manager-5665dbb77c-zzs2w"
Sep 15 21:03:10.716267 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:10.716093    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8bc5b527-c655-4c33-a3b0-cf77e0660ac3-metrics-cert\") pod \"lws-controller-manager-5665dbb77c-zzs2w\" (UID: \"8bc5b527-c655-4c33-a3b0-cf77e0660ac3\") " pod="openshift-lws-operator/lws-controller-manager-5665dbb77c-zzs2w"
Sep 15 21:03:10.716267 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:10.716155    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8bc5b527-c655-4c33-a3b0-cf77e0660ac3-cert\") pod \"lws-controller-manager-5665dbb77c-zzs2w\" (UID: \"8bc5b527-c655-4c33-a3b0-cf77e0660ac3\") " pod="openshift-lws-operator/lws-controller-manager-5665dbb77c-zzs2w"
Sep 15 21:03:10.716267 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:10.716199    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8bc5b527-c655-4c33-a3b0-cf77e0660ac3-manager-config\") pod \"lws-controller-manager-5665dbb77c-zzs2w\" (UID: \"8bc5b527-c655-4c33-a3b0-cf77e0660ac3\") " pod="openshift-lws-operator/lws-controller-manager-5665dbb77c-zzs2w"
Sep 15 21:03:10.817593 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:10.817548    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8bc5b527-c655-4c33-a3b0-cf77e0660ac3-metrics-cert\") pod \"lws-controller-manager-5665dbb77c-zzs2w\" (UID: \"8bc5b527-c655-4c33-a3b0-cf77e0660ac3\") " pod="openshift-lws-operator/lws-controller-manager-5665dbb77c-zzs2w"
Sep 15 21:03:10.817593 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:10.817596    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8bc5b527-c655-4c33-a3b0-cf77e0660ac3-cert\") pod \"lws-controller-manager-5665dbb77c-zzs2w\" (UID: \"8bc5b527-c655-4c33-a3b0-cf77e0660ac3\") " pod="openshift-lws-operator/lws-controller-manager-5665dbb77c-zzs2w"
Sep 15 21:03:10.817809 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:10.817704    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8bc5b527-c655-4c33-a3b0-cf77e0660ac3-manager-config\") pod \"lws-controller-manager-5665dbb77c-zzs2w\" (UID: \"8bc5b527-c655-4c33-a3b0-cf77e0660ac3\") " pod="openshift-lws-operator/lws-controller-manager-5665dbb77c-zzs2w"
Sep 15 21:03:10.818762 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:10.818222    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwwht\" (UniqueName: \"kubernetes.io/projected/8bc5b527-c655-4c33-a3b0-cf77e0660ac3-kube-api-access-fwwht\") pod \"lws-controller-manager-5665dbb77c-zzs2w\" (UID: \"8bc5b527-c655-4c33-a3b0-cf77e0660ac3\") " pod="openshift-lws-operator/lws-controller-manager-5665dbb77c-zzs2w"
Sep 15 21:03:10.821133 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:10.819199    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8bc5b527-c655-4c33-a3b0-cf77e0660ac3-manager-config\") pod \"lws-controller-manager-5665dbb77c-zzs2w\" (UID: \"8bc5b527-c655-4c33-a3b0-cf77e0660ac3\") " pod="openshift-lws-operator/lws-controller-manager-5665dbb77c-zzs2w"
Sep 15 21:03:10.824081 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:10.821620    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8bc5b527-c655-4c33-a3b0-cf77e0660ac3-metrics-cert\") pod \"lws-controller-manager-5665dbb77c-zzs2w\" (UID: \"8bc5b527-c655-4c33-a3b0-cf77e0660ac3\") " pod="openshift-lws-operator/lws-controller-manager-5665dbb77c-zzs2w"
Sep 15 21:03:10.824081 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:10.821747    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8bc5b527-c655-4c33-a3b0-cf77e0660ac3-cert\") pod \"lws-controller-manager-5665dbb77c-zzs2w\" (UID: \"8bc5b527-c655-4c33-a3b0-cf77e0660ac3\") " pod="openshift-lws-operator/lws-controller-manager-5665dbb77c-zzs2w"
Sep 15 21:03:10.833175 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:10.833149    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwwht\" (UniqueName: \"kubernetes.io/projected/8bc5b527-c655-4c33-a3b0-cf77e0660ac3-kube-api-access-fwwht\") pod \"lws-controller-manager-5665dbb77c-zzs2w\" (UID: \"8bc5b527-c655-4c33-a3b0-cf77e0660ac3\") " pod="openshift-lws-operator/lws-controller-manager-5665dbb77c-zzs2w"
Sep 15 21:03:10.942318 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:10.942217    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5665dbb77c-zzs2w"
Sep 15 21:03:11.257224 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:11.257196    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5665dbb77c-zzs2w"]
Sep 15 21:03:11.259860 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:03:11.259833    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bc5b527_c655_4c33_a3b0_cf77e0660ac3.slice/crio-5819507406b4dafaead170feb6177da4720c1098bfd3fcfcc1d95d8588b953cd WatchSource:0}: Error finding container 5819507406b4dafaead170feb6177da4720c1098bfd3fcfcc1d95d8588b953cd: Status 404 returned error can't find the container with id 5819507406b4dafaead170feb6177da4720c1098bfd3fcfcc1d95d8588b953cd
Sep 15 21:03:11.261915 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:11.261897    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 21:03:11.339705 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:11.339665    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:03:11.439935 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:11.439901    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:03:11.554311 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:11.554220    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5665dbb77c-zzs2w" event={"ID":"8bc5b527-c655-4c33-a3b0-cf77e0660ac3","Type":"ContainerStarted","Data":"5819507406b4dafaead170feb6177da4720c1098bfd3fcfcc1d95d8588b953cd"}
Sep 15 21:03:12.422099 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:12.422068    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:03:12.534836 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:12.534798    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:03:12.846665 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:12.846575    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:03:13.195729 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:13.195633    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:03:13.785967 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:13.785926    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:03:13.878902 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:13.878792    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:03:13.979586 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:13.979548    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:03:14.570887 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:14.570841    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5665dbb77c-zzs2w" event={"ID":"8bc5b527-c655-4c33-a3b0-cf77e0660ac3","Type":"ContainerStarted","Data":"e37f593aedb2034bebd71c1705f2646d031c1ec300a0ffaa4e0c758268a5d9f4"}
Sep 15 21:03:14.571672 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:14.571644    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5665dbb77c-zzs2w"
Sep 15 21:03:14.591222 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:14.591166    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5665dbb77c-zzs2w" podStartSLOduration=1.429663173 podStartE2EDuration="4.591147662s" totalImagesPullingTime="3.161484489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:03:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:03:14.590527234 +0000 UTC m=+331.132689868" watchObservedRunningTime="2026-09-15 21:03:14.591147662 +0000 UTC m=+331.133310297"
Sep 15 21:03:23.941342 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:23.941304    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:03:24.038712 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:24.038680    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:03:26.168723 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:26.168678    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:03:26.582124 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:03:26.582038    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5665dbb77c-zzs2w"
Sep 15 21:04:03.208086 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:03.208051    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:03.496004 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:03.495910    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:04.055472 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:04.055429    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:04.157974 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:04.157940    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:04.712091 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:04.712051    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:14.480507 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:14.480452    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:15.271752 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:15.271707    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:15.382177 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:15.382148    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:15.829718 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:15.829686    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:18.303091 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:18.303056    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-5n8hg"]
Sep 15 21:04:18.306413 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:18.306396    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-5n8hg"
Sep 15 21:04:18.309281 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:18.309258    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-kq6fp\""
Sep 15 21:04:18.309383 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:18.309357    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 15 21:04:18.309421 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:18.309379    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 15 21:04:18.331524 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:18.331475    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-5n8hg"]
Sep 15 21:04:18.382674 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:18.382640    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtxwv\" (UniqueName: \"kubernetes.io/projected/681809e4-0ba9-46be-adbd-2ee0607431d3-kube-api-access-qtxwv\") pod \"authorino-operator-76b478bd78-5n8hg\" (UID: \"681809e4-0ba9-46be-adbd-2ee0607431d3\") " pod="kuadrant-system/authorino-operator-76b478bd78-5n8hg"
Sep 15 21:04:18.483331 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:18.483293    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtxwv\" (UniqueName: \"kubernetes.io/projected/681809e4-0ba9-46be-adbd-2ee0607431d3-kube-api-access-qtxwv\") pod \"authorino-operator-76b478bd78-5n8hg\" (UID: \"681809e4-0ba9-46be-adbd-2ee0607431d3\") " pod="kuadrant-system/authorino-operator-76b478bd78-5n8hg"
Sep 15 21:04:18.498546 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:18.498519    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtxwv\" (UniqueName: \"kubernetes.io/projected/681809e4-0ba9-46be-adbd-2ee0607431d3-kube-api-access-qtxwv\") pod \"authorino-operator-76b478bd78-5n8hg\" (UID: \"681809e4-0ba9-46be-adbd-2ee0607431d3\") " pod="kuadrant-system/authorino-operator-76b478bd78-5n8hg"
Sep 15 21:04:18.616411 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:18.616372    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-5n8hg"
Sep 15 21:04:18.895125 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:18.895041    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-5n8hg"]
Sep 15 21:04:18.899092 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:04:18.899046    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod681809e4_0ba9_46be_adbd_2ee0607431d3.slice/crio-05f456658b2a045c9fad2b5d96d498439f81c223764c9f2c0b2101a6bcaa0b7b WatchSource:0}: Error finding container 05f456658b2a045c9fad2b5d96d498439f81c223764c9f2c0b2101a6bcaa0b7b: Status 404 returned error can't find the container with id 05f456658b2a045c9fad2b5d96d498439f81c223764c9f2c0b2101a6bcaa0b7b
Sep 15 21:04:19.054185 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:19.054152    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:19.287337 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:19.287241    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:19.457372 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:19.457335    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:19.854526 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:19.854456    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-5n8hg" event={"ID":"681809e4-0ba9-46be-adbd-2ee0607431d3","Type":"ContainerStarted","Data":"05f456658b2a045c9fad2b5d96d498439f81c223764c9f2c0b2101a6bcaa0b7b"}
Sep 15 21:04:20.713193 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:20.713162    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:21.225504 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:21.225452    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:21.631275 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:21.631236    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:22.043531 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:22.043434    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:22.315511 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:22.315393    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:23.874665 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:23.874623    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-5n8hg" event={"ID":"681809e4-0ba9-46be-adbd-2ee0607431d3","Type":"ContainerStarted","Data":"d66f3e0d29998ed5e04bf2cead2d9ea253a60aa1a45851c2af25200ebbfee287"}
Sep 15 21:04:23.875160 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:23.874758    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-5n8hg"
Sep 15 21:04:23.896648 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:23.896592    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-5n8hg" podStartSLOduration=1.932370225 podStartE2EDuration="5.896575052s" totalImagesPullingTime="3.964204827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:04:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:04:23.894302423 +0000 UTC m=+400.436465077" watchObservedRunningTime="2026-09-15 21:04:23.896575052 +0000 UTC m=+400.438737686"
Sep 15 21:04:25.432161 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:25.432125    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:26.865893 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:26.865854    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:28.431716 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:28.431675    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:28.528781 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:28.528746    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:28.838662 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:28.838571    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:28.941345 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:28.941310    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:32.453347 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:32.453306    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:32.550662 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:32.550625    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:34.882036 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:34.882005    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-5n8hg"
Sep 15 21:04:38.037180 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:38.037143    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:38.137923 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:38.137888    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:38.236952 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:38.236913    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:42.842330 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:42.842296    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7692t"]
Sep 15 21:04:42.845454 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:42.845438    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7692t"
Sep 15 21:04:42.849042 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:42.849020    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xslrl\""
Sep 15 21:04:42.869518 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:42.869481    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7692t"]
Sep 15 21:04:42.884891 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:42.884857    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47ad7212-94fd-47bc-8def-992e65e17196-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-7692t\" (UID: \"47ad7212-94fd-47bc-8def-992e65e17196\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7692t"
Sep 15 21:04:42.885026 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:42.884918    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xftcp\" (UniqueName: \"kubernetes.io/projected/47ad7212-94fd-47bc-8def-992e65e17196-kube-api-access-xftcp\") pod \"kuadrant-operator-controller-manager-5f685c496f-7692t\" (UID: \"47ad7212-94fd-47bc-8def-992e65e17196\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7692t"
Sep 15 21:04:42.986059 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:42.986015    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xftcp\" (UniqueName: \"kubernetes.io/projected/47ad7212-94fd-47bc-8def-992e65e17196-kube-api-access-xftcp\") pod \"kuadrant-operator-controller-manager-5f685c496f-7692t\" (UID: \"47ad7212-94fd-47bc-8def-992e65e17196\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7692t"
Sep 15 21:04:42.986269 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:42.986088    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47ad7212-94fd-47bc-8def-992e65e17196-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-7692t\" (UID: \"47ad7212-94fd-47bc-8def-992e65e17196\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7692t"
Sep 15 21:04:42.986480 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:42.986457    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47ad7212-94fd-47bc-8def-992e65e17196-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-7692t\" (UID: \"47ad7212-94fd-47bc-8def-992e65e17196\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7692t"
Sep 15 21:04:42.994549 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:42.994521    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xftcp\" (UniqueName: \"kubernetes.io/projected/47ad7212-94fd-47bc-8def-992e65e17196-kube-api-access-xftcp\") pod \"kuadrant-operator-controller-manager-5f685c496f-7692t\" (UID: \"47ad7212-94fd-47bc-8def-992e65e17196\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7692t"
Sep 15 21:04:43.155644 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:43.155612    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7692t"
Sep 15 21:04:43.430107 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:43.430023    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7692t"]
Sep 15 21:04:43.433950 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:04:43.433923    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47ad7212_94fd_47bc_8def_992e65e17196.slice/crio-1499fcbba8025b83660137d0c782e088cc4e986c41504ea523c52e07ed62be5a WatchSource:0}: Error finding container 1499fcbba8025b83660137d0c782e088cc4e986c41504ea523c52e07ed62be5a: Status 404 returned error can't find the container with id 1499fcbba8025b83660137d0c782e088cc4e986c41504ea523c52e07ed62be5a
Sep 15 21:04:43.470763 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:43.470728    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7692t"]
Sep 15 21:04:43.474835 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:43.474803    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7692t"]
Sep 15 21:04:43.498512 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:43.498464    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc"]
Sep 15 21:04:43.502968 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:43.502942    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc"
Sep 15 21:04:43.524395 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:43.524367    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc"]
Sep 15 21:04:43.592116 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:43.592079    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/856bb056-1bbd-468f-a9cd-767f70e5954d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-4nnjc\" (UID: \"856bb056-1bbd-468f-a9cd-767f70e5954d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc"
Sep 15 21:04:43.592303 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:43.592140    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8mwz\" (UniqueName: \"kubernetes.io/projected/856bb056-1bbd-468f-a9cd-767f70e5954d-kube-api-access-b8mwz\") pod \"kuadrant-operator-controller-manager-5f685c496f-4nnjc\" (UID: \"856bb056-1bbd-468f-a9cd-767f70e5954d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc"
Sep 15 21:04:43.693574 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:43.693431    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8mwz\" (UniqueName: \"kubernetes.io/projected/856bb056-1bbd-468f-a9cd-767f70e5954d-kube-api-access-b8mwz\") pod \"kuadrant-operator-controller-manager-5f685c496f-4nnjc\" (UID: \"856bb056-1bbd-468f-a9cd-767f70e5954d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc"
Sep 15 21:04:43.693574 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:43.693551    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/856bb056-1bbd-468f-a9cd-767f70e5954d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-4nnjc\" (UID: \"856bb056-1bbd-468f-a9cd-767f70e5954d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc"
Sep 15 21:04:43.693998 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:43.693974    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/856bb056-1bbd-468f-a9cd-767f70e5954d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-4nnjc\" (UID: \"856bb056-1bbd-468f-a9cd-767f70e5954d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc"
Sep 15 21:04:43.712351 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:43.712315    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8mwz\" (UniqueName: \"kubernetes.io/projected/856bb056-1bbd-468f-a9cd-767f70e5954d-kube-api-access-b8mwz\") pod \"kuadrant-operator-controller-manager-5f685c496f-4nnjc\" (UID: \"856bb056-1bbd-468f-a9cd-767f70e5954d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc"
Sep 15 21:04:43.815078 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:43.815027    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc"
Sep 15 21:04:44.098164 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:44.098138    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc"]
Sep 15 21:04:44.100545 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:04:44.100514    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod856bb056_1bbd_468f_a9cd_767f70e5954d.slice/crio-d09f1e1b5c4c481f35d0b16e5aae30b7356a3a7567bbfd00bccdafae79a09b24 WatchSource:0}: Error finding container d09f1e1b5c4c481f35d0b16e5aae30b7356a3a7567bbfd00bccdafae79a09b24: Status 404 returned error can't find the container with id d09f1e1b5c4c481f35d0b16e5aae30b7356a3a7567bbfd00bccdafae79a09b24
Sep 15 21:04:44.971574 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:44.971530    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc" event={"ID":"856bb056-1bbd-468f-a9cd-767f70e5954d","Type":"ContainerStarted","Data":"d09f1e1b5c4c481f35d0b16e5aae30b7356a3a7567bbfd00bccdafae79a09b24"}
Sep 15 21:04:45.089654 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:45.089616    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:45.527861 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:45.527620    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:46.556560 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:46.556462    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:46.654589 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:46.654479    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:48.909470 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:48.909422    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:48.996099 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:48.996042    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7692t" podUID="47ad7212-94fd-47bc-8def-992e65e17196" containerName="manager" containerID="cri-o://d4b75156556a4672a34176cb006b3a0143dbf5b5da2b7047e3755b86c9d9eb0d" gracePeriod=2
Sep 15 21:04:48.997869 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:48.997833    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc" event={"ID":"856bb056-1bbd-468f-a9cd-767f70e5954d","Type":"ContainerStarted","Data":"d84884554ad8f659a9b1b2572d4a1f0211fb891f6333e767da467fcb4588dd93"}
Sep 15 21:04:48.998033 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:48.998017    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc"
Sep 15 21:04:49.009373 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:49.009336    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:49.018809 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:49.018763    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc" podStartSLOduration=1.987584773 podStartE2EDuration="6.018746371s" totalImagesPullingTime="4.031161598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:04:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:04:49.016775159 +0000 UTC m=+425.558937814" watchObservedRunningTime="2026-09-15 21:04:49.018746371 +0000 UTC m=+425.560909075"
Sep 15 21:04:49.232568 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:49.232545    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7692t"
Sep 15 21:04:49.234931 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:04:49.234905    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-7692t\" is forbidden: User \"system:node:ip-10-0-138-221.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-221.ec2.internal' and this object" podUID="47ad7212-94fd-47bc-8def-992e65e17196" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7692t"
Sep 15 21:04:49.341869 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:49.341825    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47ad7212-94fd-47bc-8def-992e65e17196-extensions-socket-volume\") pod \"47ad7212-94fd-47bc-8def-992e65e17196\" (UID: \"47ad7212-94fd-47bc-8def-992e65e17196\") "
Sep 15 21:04:49.342026 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:49.341886    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xftcp\" (UniqueName: \"kubernetes.io/projected/47ad7212-94fd-47bc-8def-992e65e17196-kube-api-access-xftcp\") pod \"47ad7212-94fd-47bc-8def-992e65e17196\" (UID: \"47ad7212-94fd-47bc-8def-992e65e17196\") "
Sep 15 21:04:49.342188 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:49.342158    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47ad7212-94fd-47bc-8def-992e65e17196-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "47ad7212-94fd-47bc-8def-992e65e17196" (UID: "47ad7212-94fd-47bc-8def-992e65e17196"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:04:49.344292 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:49.344266    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47ad7212-94fd-47bc-8def-992e65e17196-kube-api-access-xftcp" (OuterVolumeSpecName: "kube-api-access-xftcp") pod "47ad7212-94fd-47bc-8def-992e65e17196" (UID: "47ad7212-94fd-47bc-8def-992e65e17196"). InnerVolumeSpecName "kube-api-access-xftcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:04:49.443052 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:49.442964    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/47ad7212-94fd-47bc-8def-992e65e17196-extensions-socket-volume\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:04:49.443052 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:49.442995    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xftcp\" (UniqueName: \"kubernetes.io/projected/47ad7212-94fd-47bc-8def-992e65e17196-kube-api-access-xftcp\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:04:49.499001 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:49.498961    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:49.556957 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:49.556916    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:49.975456 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:49.975426    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47ad7212-94fd-47bc-8def-992e65e17196" path="/var/lib/kubelet/pods/47ad7212-94fd-47bc-8def-992e65e17196/volumes"
Sep 15 21:04:50.003687 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:50.003652    2564 generic.go:356] "Generic (PLEG): container finished" podID="47ad7212-94fd-47bc-8def-992e65e17196" containerID="d4b75156556a4672a34176cb006b3a0143dbf5b5da2b7047e3755b86c9d9eb0d" exitCode=2
Sep 15 21:04:50.003875 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:50.003708    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7692t"
Sep 15 21:04:50.003875 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:50.003726    2564 scope.go:117] "RemoveContainer" containerID="d4b75156556a4672a34176cb006b3a0143dbf5b5da2b7047e3755b86c9d9eb0d"
Sep 15 21:04:50.008510 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:04:50.008460    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-7692t\" is forbidden: User \"system:node:ip-10-0-138-221.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-221.ec2.internal' and this object" podUID="47ad7212-94fd-47bc-8def-992e65e17196" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-7692t"
Sep 15 21:04:50.012552 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:50.012536    2564 scope.go:117] "RemoveContainer" containerID="d4b75156556a4672a34176cb006b3a0143dbf5b5da2b7047e3755b86c9d9eb0d"
Sep 15 21:04:50.012835 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:04:50.012817    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4b75156556a4672a34176cb006b3a0143dbf5b5da2b7047e3755b86c9d9eb0d\": container with ID starting with d4b75156556a4672a34176cb006b3a0143dbf5b5da2b7047e3755b86c9d9eb0d not found: ID does not exist" containerID="d4b75156556a4672a34176cb006b3a0143dbf5b5da2b7047e3755b86c9d9eb0d"
Sep 15 21:04:50.012879 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:50.012845    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4b75156556a4672a34176cb006b3a0143dbf5b5da2b7047e3755b86c9d9eb0d"} err="failed to get container status \"d4b75156556a4672a34176cb006b3a0143dbf5b5da2b7047e3755b86c9d9eb0d\": rpc error: code = NotFound desc = could not find container \"d4b75156556a4672a34176cb006b3a0143dbf5b5da2b7047e3755b86c9d9eb0d\": container with ID starting with d4b75156556a4672a34176cb006b3a0143dbf5b5da2b7047e3755b86c9d9eb0d not found: ID does not exist"
Sep 15 21:04:51.866075 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:51.866038    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:51.919714 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:51.919682    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:53.837520 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:53.837452    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:53.898196 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:53.898166    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:57.882768 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:57.882737    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:57.984298 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:57.984258    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:04:58.354432 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:04:58.354395    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:05:00.006512 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:00.006462    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc"
Sep 15 21:05:04.957302 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:04.957209    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:05:05.073922 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:05.073890    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:05:12.611446 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:12.611409    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc"]
Sep 15 21:05:12.611999 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:12.611639    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc" podUID="856bb056-1bbd-468f-a9cd-767f70e5954d" containerName="manager" containerID="cri-o://d84884554ad8f659a9b1b2572d4a1f0211fb891f6333e767da467fcb4588dd93" gracePeriod=10
Sep 15 21:05:12.868526 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:12.868448    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc"
Sep 15 21:05:12.940693 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:12.940655    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/856bb056-1bbd-468f-a9cd-767f70e5954d-extensions-socket-volume\") pod \"856bb056-1bbd-468f-a9cd-767f70e5954d\" (UID: \"856bb056-1bbd-468f-a9cd-767f70e5954d\") "
Sep 15 21:05:12.940902 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:12.940742    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8mwz\" (UniqueName: \"kubernetes.io/projected/856bb056-1bbd-468f-a9cd-767f70e5954d-kube-api-access-b8mwz\") pod \"856bb056-1bbd-468f-a9cd-767f70e5954d\" (UID: \"856bb056-1bbd-468f-a9cd-767f70e5954d\") "
Sep 15 21:05:12.941223 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:12.941193    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/856bb056-1bbd-468f-a9cd-767f70e5954d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "856bb056-1bbd-468f-a9cd-767f70e5954d" (UID: "856bb056-1bbd-468f-a9cd-767f70e5954d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:05:12.943119 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:12.943082    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/856bb056-1bbd-468f-a9cd-767f70e5954d-kube-api-access-b8mwz" (OuterVolumeSpecName: "kube-api-access-b8mwz") pod "856bb056-1bbd-468f-a9cd-767f70e5954d" (UID: "856bb056-1bbd-468f-a9cd-767f70e5954d"). InnerVolumeSpecName "kube-api-access-b8mwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:05:13.042139 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:13.042091    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8mwz\" (UniqueName: \"kubernetes.io/projected/856bb056-1bbd-468f-a9cd-767f70e5954d-kube-api-access-b8mwz\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:05:13.042139 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:13.042129    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/856bb056-1bbd-468f-a9cd-767f70e5954d-extensions-socket-volume\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:05:13.119659 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:13.119575    2564 generic.go:356] "Generic (PLEG): container finished" podID="856bb056-1bbd-468f-a9cd-767f70e5954d" containerID="d84884554ad8f659a9b1b2572d4a1f0211fb891f6333e767da467fcb4588dd93" exitCode=0
Sep 15 21:05:13.119659 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:13.119612    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc" event={"ID":"856bb056-1bbd-468f-a9cd-767f70e5954d","Type":"ContainerDied","Data":"d84884554ad8f659a9b1b2572d4a1f0211fb891f6333e767da467fcb4588dd93"}
Sep 15 21:05:13.119659 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:13.119627    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc"
Sep 15 21:05:13.119659 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:13.119652    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc" event={"ID":"856bb056-1bbd-468f-a9cd-767f70e5954d","Type":"ContainerDied","Data":"d09f1e1b5c4c481f35d0b16e5aae30b7356a3a7567bbfd00bccdafae79a09b24"}
Sep 15 21:05:13.119914 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:13.119673    2564 scope.go:117] "RemoveContainer" containerID="d84884554ad8f659a9b1b2572d4a1f0211fb891f6333e767da467fcb4588dd93"
Sep 15 21:05:13.129062 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:13.129044    2564 scope.go:117] "RemoveContainer" containerID="d84884554ad8f659a9b1b2572d4a1f0211fb891f6333e767da467fcb4588dd93"
Sep 15 21:05:13.129356 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:05:13.129332    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d84884554ad8f659a9b1b2572d4a1f0211fb891f6333e767da467fcb4588dd93\": container with ID starting with d84884554ad8f659a9b1b2572d4a1f0211fb891f6333e767da467fcb4588dd93 not found: ID does not exist" containerID="d84884554ad8f659a9b1b2572d4a1f0211fb891f6333e767da467fcb4588dd93"
Sep 15 21:05:13.129410 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:13.129366    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d84884554ad8f659a9b1b2572d4a1f0211fb891f6333e767da467fcb4588dd93"} err="failed to get container status \"d84884554ad8f659a9b1b2572d4a1f0211fb891f6333e767da467fcb4588dd93\": rpc error: code = NotFound desc = could not find container \"d84884554ad8f659a9b1b2572d4a1f0211fb891f6333e767da467fcb4588dd93\": container with ID starting with d84884554ad8f659a9b1b2572d4a1f0211fb891f6333e767da467fcb4588dd93 not found: ID does not exist"
Sep 15 21:05:13.142139 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:13.142102    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc"]
Sep 15 21:05:13.145781 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:13.145756    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4nnjc"]
Sep 15 21:05:13.972345 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:13.972313    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="856bb056-1bbd-468f-a9cd-767f70e5954d" path="/var/lib/kubelet/pods/856bb056-1bbd-468f-a9cd-767f70e5954d/volumes"
Sep 15 21:05:16.545365 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:16.545323    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:05:16.742364 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:16.742322    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:05:31.578602 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:31.578553    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cnqt7"]
Sep 15 21:05:31.579014 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:31.578916    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47ad7212-94fd-47bc-8def-992e65e17196" containerName="manager"
Sep 15 21:05:31.579014 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:31.578927    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="47ad7212-94fd-47bc-8def-992e65e17196" containerName="manager"
Sep 15 21:05:31.579014 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:31.578935    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="856bb056-1bbd-468f-a9cd-767f70e5954d" containerName="manager"
Sep 15 21:05:31.579014 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:31.578941    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="856bb056-1bbd-468f-a9cd-767f70e5954d" containerName="manager"
Sep 15 21:05:31.579014 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:31.579005    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="47ad7212-94fd-47bc-8def-992e65e17196" containerName="manager"
Sep 15 21:05:31.579014 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:31.579011    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="856bb056-1bbd-468f-a9cd-767f70e5954d" containerName="manager"
Sep 15 21:05:31.581173 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:31.581141    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:05:31.582550 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:31.582535    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-cnqt7"
Sep 15 21:05:31.585109 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:31.585084    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 15 21:05:31.585228 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:31.585206    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-85rzs\""
Sep 15 21:05:31.591544 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:31.591519    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cnqt7"]
Sep 15 21:05:31.680941 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:31.680908    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cnqt7"]
Sep 15 21:05:31.692253 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:31.692197    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:05:31.709720 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:31.709686    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2-config-file\") pod \"limitador-limitador-8588749977-cnqt7\" (UID: \"08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2\") " pod="kuadrant-system/limitador-limitador-8588749977-cnqt7"
Sep 15 21:05:31.709720 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:31.709721    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t99qd\" (UniqueName: \"kubernetes.io/projected/08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2-kube-api-access-t99qd\") pod \"limitador-limitador-8588749977-cnqt7\" (UID: \"08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2\") " pod="kuadrant-system/limitador-limitador-8588749977-cnqt7"
Sep 15 21:05:31.793241 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:31.793202    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:05:31.810986 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:31.810950    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2-config-file\") pod \"limitador-limitador-8588749977-cnqt7\" (UID: \"08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2\") " pod="kuadrant-system/limitador-limitador-8588749977-cnqt7"
Sep 15 21:05:31.810986 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:31.810986    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t99qd\" (UniqueName: \"kubernetes.io/projected/08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2-kube-api-access-t99qd\") pod \"limitador-limitador-8588749977-cnqt7\" (UID: \"08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2\") " pod="kuadrant-system/limitador-limitador-8588749977-cnqt7"
Sep 15 21:05:31.811683 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:31.811661    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2-config-file\") pod \"limitador-limitador-8588749977-cnqt7\" (UID: \"08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2\") " pod="kuadrant-system/limitador-limitador-8588749977-cnqt7"
Sep 15 21:05:31.824982 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:31.824945    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t99qd\" (UniqueName: \"kubernetes.io/projected/08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2-kube-api-access-t99qd\") pod \"limitador-limitador-8588749977-cnqt7\" (UID: \"08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2\") " pod="kuadrant-system/limitador-limitador-8588749977-cnqt7"
Sep 15 21:05:31.893020 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:31.892964    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-cnqt7"
Sep 15 21:05:32.178228 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:32.178150    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cnqt7"]
Sep 15 21:05:32.181161 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:05:32.181130    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08d22e06_5ef7_4e7f_90eb_ae3f25d6d5f2.slice/crio-b2e059013340cf9fe35ca9d4ddc756fd120f0b95eb466c2b7df99b474c7682da WatchSource:0}: Error finding container b2e059013340cf9fe35ca9d4ddc756fd120f0b95eb466c2b7df99b474c7682da: Status 404 returned error can't find the container with id b2e059013340cf9fe35ca9d4ddc756fd120f0b95eb466c2b7df99b474c7682da
Sep 15 21:05:32.211234 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:32.211198    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-cnqt7" event={"ID":"08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2","Type":"ContainerStarted","Data":"b2e059013340cf9fe35ca9d4ddc756fd120f0b95eb466c2b7df99b474c7682da"}
Sep 15 21:05:32.630407 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:32.630372    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:05:32.732690 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:32.732657    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:05:34.221002 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:34.220966    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-cnqt7" event={"ID":"08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2","Type":"ContainerStarted","Data":"27b033e5f23db2b6210aba7d618fc5bace7461cc80885d74bf040be04abedf94"}
Sep 15 21:05:34.221412 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:34.221041    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-cnqt7"
Sep 15 21:05:34.241319 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:34.241262    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-cnqt7" podStartSLOduration=1.8211441819999998 podStartE2EDuration="3.241245142s" totalImagesPullingTime="1.42010096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:05:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:05:34.240101647 +0000 UTC m=+470.782264304" watchObservedRunningTime="2026-09-15 21:05:34.241245142 +0000 UTC m=+470.783407824"
Sep 15 21:05:43.239139 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:43.239096    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:05:44.540062 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:44.540028    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:05:45.227126 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:45.227099    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-cnqt7"
Sep 15 21:05:47.176041 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:47.176001    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cnqt7"]
Sep 15 21:05:47.176474 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:47.176189    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-cnqt7" podUID="08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2" containerName="limitador" containerID="cri-o://27b033e5f23db2b6210aba7d618fc5bace7461cc80885d74bf040be04abedf94" gracePeriod=30
Sep 15 21:05:48.121367 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:48.121339    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-cnqt7"
Sep 15 21:05:48.127564 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:48.127542    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t99qd\" (UniqueName: \"kubernetes.io/projected/08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2-kube-api-access-t99qd\") pod \"08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2\" (UID: \"08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2\") "
Sep 15 21:05:48.127661 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:48.127600    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2-config-file\") pod \"08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2\" (UID: \"08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2\") "
Sep 15 21:05:48.127976 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:48.127952    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2-config-file" (OuterVolumeSpecName: "config-file") pod "08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2" (UID: "08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:05:48.129898 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:48.129874    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2-kube-api-access-t99qd" (OuterVolumeSpecName: "kube-api-access-t99qd") pod "08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2" (UID: "08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2"). InnerVolumeSpecName "kube-api-access-t99qd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:05:48.228916 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:48.228872    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t99qd\" (UniqueName: \"kubernetes.io/projected/08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2-kube-api-access-t99qd\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:05:48.228916 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:48.228909    2564 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2-config-file\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:05:48.286204 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:48.286119    2564 generic.go:356] "Generic (PLEG): container finished" podID="08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2" containerID="27b033e5f23db2b6210aba7d618fc5bace7461cc80885d74bf040be04abedf94" exitCode=0
Sep 15 21:05:48.286204 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:48.286162    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-cnqt7" event={"ID":"08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2","Type":"ContainerDied","Data":"27b033e5f23db2b6210aba7d618fc5bace7461cc80885d74bf040be04abedf94"}
Sep 15 21:05:48.286387 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:48.286207    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-cnqt7" event={"ID":"08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2","Type":"ContainerDied","Data":"b2e059013340cf9fe35ca9d4ddc756fd120f0b95eb466c2b7df99b474c7682da"}
Sep 15 21:05:48.286387 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:48.286230    2564 scope.go:117] "RemoveContainer" containerID="27b033e5f23db2b6210aba7d618fc5bace7461cc80885d74bf040be04abedf94"
Sep 15 21:05:48.286387 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:48.286163    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-cnqt7"
Sep 15 21:05:48.295342 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:48.295322    2564 scope.go:117] "RemoveContainer" containerID="27b033e5f23db2b6210aba7d618fc5bace7461cc80885d74bf040be04abedf94"
Sep 15 21:05:48.295674 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:05:48.295652    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27b033e5f23db2b6210aba7d618fc5bace7461cc80885d74bf040be04abedf94\": container with ID starting with 27b033e5f23db2b6210aba7d618fc5bace7461cc80885d74bf040be04abedf94 not found: ID does not exist" containerID="27b033e5f23db2b6210aba7d618fc5bace7461cc80885d74bf040be04abedf94"
Sep 15 21:05:48.295741 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:48.295684    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27b033e5f23db2b6210aba7d618fc5bace7461cc80885d74bf040be04abedf94"} err="failed to get container status \"27b033e5f23db2b6210aba7d618fc5bace7461cc80885d74bf040be04abedf94\": rpc error: code = NotFound desc = could not find container \"27b033e5f23db2b6210aba7d618fc5bace7461cc80885d74bf040be04abedf94\": container with ID starting with 27b033e5f23db2b6210aba7d618fc5bace7461cc80885d74bf040be04abedf94 not found: ID does not exist"
Sep 15 21:05:48.313507 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:48.313456    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cnqt7"]
Sep 15 21:05:48.318100 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:48.318069    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-cnqt7"]
Sep 15 21:05:49.971934 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:49.971901    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2" path="/var/lib/kubelet/pods/08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2/volumes"
Sep 15 21:05:50.227186 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:50.227091    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:05:50.325253 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:50.325220    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:05:50.662875 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:50.662844    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wkqcs"]
Sep 15 21:05:50.663160 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:50.663147    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2" containerName="limitador"
Sep 15 21:05:50.663226 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:50.663162    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2" containerName="limitador"
Sep 15 21:05:50.663280 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:50.663235    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="08d22e06-5ef7-4e7f-90eb-ae3f25d6d5f2" containerName="limitador"
Sep 15 21:05:50.665829 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:50.665808    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wkqcs"
Sep 15 21:05:50.668790 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:50.668764    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 15 21:05:50.668920 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:50.668787    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 21:05:50.668920 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:50.668829    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 15 21:05:50.670047 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:50.670018    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-6g7qw\""
Sep 15 21:05:50.675777 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:50.675751    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wkqcs"]
Sep 15 21:05:50.701843 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:50.701805    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:05:50.749511 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:50.749445    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d6bfdde1-a751-4785-aa00-00348f766cc9-data\") pod \"postgres-69fb4594cf-wkqcs\" (UID: \"d6bfdde1-a751-4785-aa00-00348f766cc9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wkqcs"
Sep 15 21:05:50.749511 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:50.749483    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhtkw\" (UniqueName: \"kubernetes.io/projected/d6bfdde1-a751-4785-aa00-00348f766cc9-kube-api-access-lhtkw\") pod \"postgres-69fb4594cf-wkqcs\" (UID: \"d6bfdde1-a751-4785-aa00-00348f766cc9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wkqcs"
Sep 15 21:05:50.850721 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:50.850680    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d6bfdde1-a751-4785-aa00-00348f766cc9-data\") pod \"postgres-69fb4594cf-wkqcs\" (UID: \"d6bfdde1-a751-4785-aa00-00348f766cc9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wkqcs"
Sep 15 21:05:50.850721 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:50.850723    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhtkw\" (UniqueName: \"kubernetes.io/projected/d6bfdde1-a751-4785-aa00-00348f766cc9-kube-api-access-lhtkw\") pod \"postgres-69fb4594cf-wkqcs\" (UID: \"d6bfdde1-a751-4785-aa00-00348f766cc9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wkqcs"
Sep 15 21:05:50.851088 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:50.851067    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d6bfdde1-a751-4785-aa00-00348f766cc9-data\") pod \"postgres-69fb4594cf-wkqcs\" (UID: \"d6bfdde1-a751-4785-aa00-00348f766cc9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wkqcs"
Sep 15 21:05:50.859183 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:50.859161    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhtkw\" (UniqueName: \"kubernetes.io/projected/d6bfdde1-a751-4785-aa00-00348f766cc9-kube-api-access-lhtkw\") pod \"postgres-69fb4594cf-wkqcs\" (UID: \"d6bfdde1-a751-4785-aa00-00348f766cc9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wkqcs"
Sep 15 21:05:50.978021 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:50.977919    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wkqcs"
Sep 15 21:05:51.251245 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:51.251111    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wkqcs"]
Sep 15 21:05:51.254069 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:05:51.254042    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6bfdde1_a751_4785_aa00_00348f766cc9.slice/crio-4b76299676e818ac1dae2c58e40fe8b09ad56ccaf8254c5c67bc0f3481e49457 WatchSource:0}: Error finding container 4b76299676e818ac1dae2c58e40fe8b09ad56ccaf8254c5c67bc0f3481e49457: Status 404 returned error can't find the container with id 4b76299676e818ac1dae2c58e40fe8b09ad56ccaf8254c5c67bc0f3481e49457
Sep 15 21:05:51.302993 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:51.302953    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wkqcs" event={"ID":"d6bfdde1-a751-4785-aa00-00348f766cc9","Type":"ContainerStarted","Data":"4b76299676e818ac1dae2c58e40fe8b09ad56ccaf8254c5c67bc0f3481e49457"}
Sep 15 21:05:57.339326 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:57.339283    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wkqcs" event={"ID":"d6bfdde1-a751-4785-aa00-00348f766cc9","Type":"ContainerStarted","Data":"a6e1592830b0197047ec5e0f728e5e2f2c844fd700feae3ff945cbe765df1b7c"}
Sep 15 21:05:57.339746 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:57.339374    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wkqcs"
Sep 15 21:05:57.356123 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:05:57.356033    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wkqcs" podStartSLOduration=2.335643804 podStartE2EDuration="7.35601454s" totalImagesPullingTime="5.020370736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:05:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:05:57.35482029 +0000 UTC m=+493.896982948" watchObservedRunningTime="2026-09-15 21:05:57.35601454 +0000 UTC m=+493.898177175"
Sep 15 21:06:03.372613 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:03.372584    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wkqcs"
Sep 15 21:06:03.723049 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:03.722957    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:06:03.941541 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:03.941482    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-57njq"]
Sep 15 21:06:03.944412 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:03.944394    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-57njq"
Sep 15 21:06:03.946876 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:03.946855    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2z26s\""
Sep 15 21:06:03.952402 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:03.952373    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-57njq"]
Sep 15 21:06:04.079913 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.079821    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctrq7\" (UniqueName: \"kubernetes.io/projected/2d9c59fc-c16e-462b-bf24-c69bdfbea45e-kube-api-access-ctrq7\") pod \"authorino-557f876f69-57njq\" (UID: \"2d9c59fc-c16e-462b-bf24-c69bdfbea45e\") " pod="kuadrant-system/authorino-557f876f69-57njq"
Sep 15 21:06:04.140818 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.140780    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-57njq"]
Sep 15 21:06:04.141058 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:06:04.141038    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-ctrq7], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-57njq" podUID="2d9c59fc-c16e-462b-bf24-c69bdfbea45e"
Sep 15 21:06:04.167343 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.167310    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-ptdt9"]
Sep 15 21:06:04.169473 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.169456    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-ptdt9"
Sep 15 21:06:04.171969 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.171945    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 15 21:06:04.179075 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.179052    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-ptdt9"]
Sep 15 21:06:04.180518 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.180477    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctrq7\" (UniqueName: \"kubernetes.io/projected/2d9c59fc-c16e-462b-bf24-c69bdfbea45e-kube-api-access-ctrq7\") pod \"authorino-557f876f69-57njq\" (UID: \"2d9c59fc-c16e-462b-bf24-c69bdfbea45e\") " pod="kuadrant-system/authorino-557f876f69-57njq"
Sep 15 21:06:04.191669 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.191630    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctrq7\" (UniqueName: \"kubernetes.io/projected/2d9c59fc-c16e-462b-bf24-c69bdfbea45e-kube-api-access-ctrq7\") pod \"authorino-557f876f69-57njq\" (UID: \"2d9c59fc-c16e-462b-bf24-c69bdfbea45e\") " pod="kuadrant-system/authorino-557f876f69-57njq"
Sep 15 21:06:04.199039 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.199012    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-ptdt9"]
Sep 15 21:06:04.199270 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:06:04.199253    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-vxskz tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-vxskz tls-cert]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-ptdt9" podUID="48c0d073-3fd4-4f19-a07d-a5492ea2211d"
Sep 15 21:06:04.236817 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.236787    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:06:04.281113 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.281080    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/48c0d073-3fd4-4f19-a07d-a5492ea2211d-tls-cert\") pod \"authorino-6d4dfc649f-ptdt9\" (UID: \"48c0d073-3fd4-4f19-a07d-a5492ea2211d\") " pod="kuadrant-system/authorino-6d4dfc649f-ptdt9"
Sep 15 21:06:04.281308 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.281132    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxskz\" (UniqueName: \"kubernetes.io/projected/48c0d073-3fd4-4f19-a07d-a5492ea2211d-kube-api-access-vxskz\") pod \"authorino-6d4dfc649f-ptdt9\" (UID: \"48c0d073-3fd4-4f19-a07d-a5492ea2211d\") " pod="kuadrant-system/authorino-6d4dfc649f-ptdt9"
Sep 15 21:06:04.370285 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.370258    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-57njq"
Sep 15 21:06:04.370455 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.370258    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-ptdt9"
Sep 15 21:06:04.376691 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.376667    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-57njq"
Sep 15 21:06:04.379752 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.379735    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-ptdt9"
Sep 15 21:06:04.382075 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.382057    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/48c0d073-3fd4-4f19-a07d-a5492ea2211d-tls-cert\") pod \"authorino-6d4dfc649f-ptdt9\" (UID: \"48c0d073-3fd4-4f19-a07d-a5492ea2211d\") " pod="kuadrant-system/authorino-6d4dfc649f-ptdt9"
Sep 15 21:06:04.382118 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.382095    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxskz\" (UniqueName: \"kubernetes.io/projected/48c0d073-3fd4-4f19-a07d-a5492ea2211d-kube-api-access-vxskz\") pod \"authorino-6d4dfc649f-ptdt9\" (UID: \"48c0d073-3fd4-4f19-a07d-a5492ea2211d\") " pod="kuadrant-system/authorino-6d4dfc649f-ptdt9"
Sep 15 21:06:04.384840 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.384819    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/48c0d073-3fd4-4f19-a07d-a5492ea2211d-tls-cert\") pod \"authorino-6d4dfc649f-ptdt9\" (UID: \"48c0d073-3fd4-4f19-a07d-a5492ea2211d\") " pod="kuadrant-system/authorino-6d4dfc649f-ptdt9"
Sep 15 21:06:04.390436 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.390414    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxskz\" (UniqueName: \"kubernetes.io/projected/48c0d073-3fd4-4f19-a07d-a5492ea2211d-kube-api-access-vxskz\") pod \"authorino-6d4dfc649f-ptdt9\" (UID: \"48c0d073-3fd4-4f19-a07d-a5492ea2211d\") " pod="kuadrant-system/authorino-6d4dfc649f-ptdt9"
Sep 15 21:06:04.483296 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.483254    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctrq7\" (UniqueName: \"kubernetes.io/projected/2d9c59fc-c16e-462b-bf24-c69bdfbea45e-kube-api-access-ctrq7\") pod \"2d9c59fc-c16e-462b-bf24-c69bdfbea45e\" (UID: \"2d9c59fc-c16e-462b-bf24-c69bdfbea45e\") "
Sep 15 21:06:04.483296 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.483302    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/48c0d073-3fd4-4f19-a07d-a5492ea2211d-tls-cert\") pod \"48c0d073-3fd4-4f19-a07d-a5492ea2211d\" (UID: \"48c0d073-3fd4-4f19-a07d-a5492ea2211d\") "
Sep 15 21:06:04.483571 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.483387    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxskz\" (UniqueName: \"kubernetes.io/projected/48c0d073-3fd4-4f19-a07d-a5492ea2211d-kube-api-access-vxskz\") pod \"48c0d073-3fd4-4f19-a07d-a5492ea2211d\" (UID: \"48c0d073-3fd4-4f19-a07d-a5492ea2211d\") "
Sep 15 21:06:04.485707 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.485673    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d9c59fc-c16e-462b-bf24-c69bdfbea45e-kube-api-access-ctrq7" (OuterVolumeSpecName: "kube-api-access-ctrq7") pod "2d9c59fc-c16e-462b-bf24-c69bdfbea45e" (UID: "2d9c59fc-c16e-462b-bf24-c69bdfbea45e"). InnerVolumeSpecName "kube-api-access-ctrq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:06:04.485825 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.485757    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48c0d073-3fd4-4f19-a07d-a5492ea2211d-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "48c0d073-3fd4-4f19-a07d-a5492ea2211d" (UID: "48c0d073-3fd4-4f19-a07d-a5492ea2211d"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:06:04.485825 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.485766    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48c0d073-3fd4-4f19-a07d-a5492ea2211d-kube-api-access-vxskz" (OuterVolumeSpecName: "kube-api-access-vxskz") pod "48c0d073-3fd4-4f19-a07d-a5492ea2211d" (UID: "48c0d073-3fd4-4f19-a07d-a5492ea2211d"). InnerVolumeSpecName "kube-api-access-vxskz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:06:04.584076 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.584040    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxskz\" (UniqueName: \"kubernetes.io/projected/48c0d073-3fd4-4f19-a07d-a5492ea2211d-kube-api-access-vxskz\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:06:04.584076 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.584072    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctrq7\" (UniqueName: \"kubernetes.io/projected/2d9c59fc-c16e-462b-bf24-c69bdfbea45e-kube-api-access-ctrq7\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:06:04.584076 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:04.584080    2564 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/48c0d073-3fd4-4f19-a07d-a5492ea2211d-tls-cert\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:06:05.374100 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:05.374059    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-ptdt9"
Sep 15 21:06:05.374317 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:05.374061    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-57njq"
Sep 15 21:06:05.405791 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:05.405753    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-57njq"]
Sep 15 21:06:05.409641 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:05.409614    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-57njq"]
Sep 15 21:06:05.433227 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:05.433193    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-ptdt9"]
Sep 15 21:06:05.435570 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:05.435546    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-ptdt9"]
Sep 15 21:06:05.972183 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:05.972153    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d9c59fc-c16e-462b-bf24-c69bdfbea45e" path="/var/lib/kubelet/pods/2d9c59fc-c16e-462b-bf24-c69bdfbea45e/volumes"
Sep 15 21:06:05.972410 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:05.972398    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48c0d073-3fd4-4f19-a07d-a5492ea2211d" path="/var/lib/kubelet/pods/48c0d073-3fd4-4f19-a07d-a5492ea2211d/volumes"
Sep 15 21:06:11.318031 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.317997    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:06:11.340541 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.340512    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-84f9876df9-z9cj7"]
Sep 15 21:06:11.369509 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.369456    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84f9876df9-z9cj7"]
Sep 15 21:06:11.369687 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.369598    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84f9876df9-z9cj7"
Sep 15 21:06:11.372839 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.372808    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 15 21:06:11.372839 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.372826    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 15 21:06:11.374017 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.373997    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lsxq7\""
Sep 15 21:06:11.374197 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.374043    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 15 21:06:11.390819 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.390784    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:06:11.442767 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.442733    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cc9l\" (UniqueName: \"kubernetes.io/projected/e4686b9b-c253-4dcd-bb87-35a1924cd523-kube-api-access-6cc9l\") pod \"maas-controller-84f9876df9-z9cj7\" (UID: \"e4686b9b-c253-4dcd-bb87-35a1924cd523\") " pod="opendatahub/maas-controller-84f9876df9-z9cj7"
Sep 15 21:06:11.442949 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.442793    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4686b9b-c253-4dcd-bb87-35a1924cd523-metrics-certs\") pod \"maas-controller-84f9876df9-z9cj7\" (UID: \"e4686b9b-c253-4dcd-bb87-35a1924cd523\") " pod="opendatahub/maas-controller-84f9876df9-z9cj7"
Sep 15 21:06:11.442949 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.442844    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4686b9b-c253-4dcd-bb87-35a1924cd523-webhook-cert\") pod \"maas-controller-84f9876df9-z9cj7\" (UID: \"e4686b9b-c253-4dcd-bb87-35a1924cd523\") " pod="opendatahub/maas-controller-84f9876df9-z9cj7"
Sep 15 21:06:11.444692 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.444651    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:06:11.520867 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.520822    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:06:11.525232 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.525205    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7b6486456d-h4hnf"]
Sep 15 21:06:11.527878 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.527864    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b6486456d-h4hnf"
Sep 15 21:06:11.539708 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.539682    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b6486456d-h4hnf"]
Sep 15 21:06:11.543442 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.543419    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4686b9b-c253-4dcd-bb87-35a1924cd523-webhook-cert\") pod \"maas-controller-84f9876df9-z9cj7\" (UID: \"e4686b9b-c253-4dcd-bb87-35a1924cd523\") " pod="opendatahub/maas-controller-84f9876df9-z9cj7"
Sep 15 21:06:11.543570 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.543482    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cc9l\" (UniqueName: \"kubernetes.io/projected/e4686b9b-c253-4dcd-bb87-35a1924cd523-kube-api-access-6cc9l\") pod \"maas-controller-84f9876df9-z9cj7\" (UID: \"e4686b9b-c253-4dcd-bb87-35a1924cd523\") " pod="opendatahub/maas-controller-84f9876df9-z9cj7"
Sep 15 21:06:11.543613 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.543580    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4686b9b-c253-4dcd-bb87-35a1924cd523-metrics-certs\") pod \"maas-controller-84f9876df9-z9cj7\" (UID: \"e4686b9b-c253-4dcd-bb87-35a1924cd523\") " pod="opendatahub/maas-controller-84f9876df9-z9cj7"
Sep 15 21:06:11.546187 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.546165    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4686b9b-c253-4dcd-bb87-35a1924cd523-webhook-cert\") pod \"maas-controller-84f9876df9-z9cj7\" (UID: \"e4686b9b-c253-4dcd-bb87-35a1924cd523\") " pod="opendatahub/maas-controller-84f9876df9-z9cj7"
Sep 15 21:06:11.546279 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.546220    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4686b9b-c253-4dcd-bb87-35a1924cd523-metrics-certs\") pod \"maas-controller-84f9876df9-z9cj7\" (UID: \"e4686b9b-c253-4dcd-bb87-35a1924cd523\") " pod="opendatahub/maas-controller-84f9876df9-z9cj7"
Sep 15 21:06:11.551303 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.551279    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cc9l\" (UniqueName: \"kubernetes.io/projected/e4686b9b-c253-4dcd-bb87-35a1924cd523-kube-api-access-6cc9l\") pod \"maas-controller-84f9876df9-z9cj7\" (UID: \"e4686b9b-c253-4dcd-bb87-35a1924cd523\") " pod="opendatahub/maas-controller-84f9876df9-z9cj7"
Sep 15 21:06:11.582985 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.582893    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:06:11.644964 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.644928    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a8b7760f-2f30-4bd8-b87e-afeb2cd908b9-webhook-cert\") pod \"maas-controller-7b6486456d-h4hnf\" (UID: \"a8b7760f-2f30-4bd8-b87e-afeb2cd908b9\") " pod="opendatahub/maas-controller-7b6486456d-h4hnf"
Sep 15 21:06:11.644964 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.644969    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vz8p\" (UniqueName: \"kubernetes.io/projected/a8b7760f-2f30-4bd8-b87e-afeb2cd908b9-kube-api-access-7vz8p\") pod \"maas-controller-7b6486456d-h4hnf\" (UID: \"a8b7760f-2f30-4bd8-b87e-afeb2cd908b9\") " pod="opendatahub/maas-controller-7b6486456d-h4hnf"
Sep 15 21:06:11.645169 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.645044    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8b7760f-2f30-4bd8-b87e-afeb2cd908b9-metrics-certs\") pod \"maas-controller-7b6486456d-h4hnf\" (UID: \"a8b7760f-2f30-4bd8-b87e-afeb2cd908b9\") " pod="opendatahub/maas-controller-7b6486456d-h4hnf"
Sep 15 21:06:11.681912 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.681889    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84f9876df9-z9cj7"
Sep 15 21:06:11.746193 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.746124    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a8b7760f-2f30-4bd8-b87e-afeb2cd908b9-webhook-cert\") pod \"maas-controller-7b6486456d-h4hnf\" (UID: \"a8b7760f-2f30-4bd8-b87e-afeb2cd908b9\") " pod="opendatahub/maas-controller-7b6486456d-h4hnf"
Sep 15 21:06:11.746363 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.746216    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vz8p\" (UniqueName: \"kubernetes.io/projected/a8b7760f-2f30-4bd8-b87e-afeb2cd908b9-kube-api-access-7vz8p\") pod \"maas-controller-7b6486456d-h4hnf\" (UID: \"a8b7760f-2f30-4bd8-b87e-afeb2cd908b9\") " pod="opendatahub/maas-controller-7b6486456d-h4hnf"
Sep 15 21:06:11.746363 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.746297    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8b7760f-2f30-4bd8-b87e-afeb2cd908b9-metrics-certs\") pod \"maas-controller-7b6486456d-h4hnf\" (UID: \"a8b7760f-2f30-4bd8-b87e-afeb2cd908b9\") " pod="opendatahub/maas-controller-7b6486456d-h4hnf"
Sep 15 21:06:11.750233 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.750203    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a8b7760f-2f30-4bd8-b87e-afeb2cd908b9-webhook-cert\") pod \"maas-controller-7b6486456d-h4hnf\" (UID: \"a8b7760f-2f30-4bd8-b87e-afeb2cd908b9\") " pod="opendatahub/maas-controller-7b6486456d-h4hnf"
Sep 15 21:06:11.750693 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.750671    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8b7760f-2f30-4bd8-b87e-afeb2cd908b9-metrics-certs\") pod \"maas-controller-7b6486456d-h4hnf\" (UID: \"a8b7760f-2f30-4bd8-b87e-afeb2cd908b9\") " pod="opendatahub/maas-controller-7b6486456d-h4hnf"
Sep 15 21:06:11.758938 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.758909    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vz8p\" (UniqueName: \"kubernetes.io/projected/a8b7760f-2f30-4bd8-b87e-afeb2cd908b9-kube-api-access-7vz8p\") pod \"maas-controller-7b6486456d-h4hnf\" (UID: \"a8b7760f-2f30-4bd8-b87e-afeb2cd908b9\") " pod="opendatahub/maas-controller-7b6486456d-h4hnf"
Sep 15 21:06:11.837720 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.837633    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b6486456d-h4hnf"
Sep 15 21:06:11.951971 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:11.951943    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84f9876df9-z9cj7"]
Sep 15 21:06:11.953635 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:06:11.953607    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4686b9b_c253_4dcd_bb87_35a1924cd523.slice/crio-7536766bcd9666828c2cb9753e371f71028fa15f833462f4d854083e434c770e WatchSource:0}: Error finding container 7536766bcd9666828c2cb9753e371f71028fa15f833462f4d854083e434c770e: Status 404 returned error can't find the container with id 7536766bcd9666828c2cb9753e371f71028fa15f833462f4d854083e434c770e
Sep 15 21:06:12.110140 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:12.110026    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b6486456d-h4hnf"]
Sep 15 21:06:12.112782 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:06:12.112753    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8b7760f_2f30_4bd8_b87e_afeb2cd908b9.slice/crio-f6dcc144a57f0363b9c2e85eb031d2ede0f03990713da0d0750cd4fd32b38c08 WatchSource:0}: Error finding container f6dcc144a57f0363b9c2e85eb031d2ede0f03990713da0d0750cd4fd32b38c08: Status 404 returned error can't find the container with id f6dcc144a57f0363b9c2e85eb031d2ede0f03990713da0d0750cd4fd32b38c08
Sep 15 21:06:12.406365 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:12.406264    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b6486456d-h4hnf" event={"ID":"a8b7760f-2f30-4bd8-b87e-afeb2cd908b9","Type":"ContainerStarted","Data":"f6dcc144a57f0363b9c2e85eb031d2ede0f03990713da0d0750cd4fd32b38c08"}
Sep 15 21:06:12.407464 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:12.407438    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84f9876df9-z9cj7" event={"ID":"e4686b9b-c253-4dcd-bb87-35a1924cd523","Type":"ContainerStarted","Data":"7536766bcd9666828c2cb9753e371f71028fa15f833462f4d854083e434c770e"}
Sep 15 21:06:14.801673 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:14.801630    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:06:14.932457 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:14.932362    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:06:15.428109 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:15.428066    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b6486456d-h4hnf" event={"ID":"a8b7760f-2f30-4bd8-b87e-afeb2cd908b9","Type":"ContainerStarted","Data":"514c068f4887036125ca3d13f50e6392b5bbcf27363e9f5698235db63dc7cf88"}
Sep 15 21:06:15.428317 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:15.428215    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7b6486456d-h4hnf"
Sep 15 21:06:15.429722 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:15.429694    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84f9876df9-z9cj7" event={"ID":"e4686b9b-c253-4dcd-bb87-35a1924cd523","Type":"ContainerStarted","Data":"7dd9f6e48b97c33ee6af2a595073aea4efa3b8b40b9b9d2581467705c8ef076e"}
Sep 15 21:06:15.429857 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:15.429820    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-84f9876df9-z9cj7"
Sep 15 21:06:15.447669 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:15.447614    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7b6486456d-h4hnf" podStartSLOduration=1.923524967 podStartE2EDuration="4.447596885s" totalImagesPullingTime="2.524071918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:06:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:06:15.445935184 +0000 UTC m=+511.988097872" watchObservedRunningTime="2026-09-15 21:06:15.447596885 +0000 UTC m=+511.989759584"
Sep 15 21:06:15.463626 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:15.463572    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-84f9876df9-z9cj7" podStartSLOduration=1.7898278319999998 podStartE2EDuration="4.463552946s" totalImagesPullingTime="2.673725114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:06:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:06:15.463006401 +0000 UTC m=+512.005169055" watchObservedRunningTime="2026-09-15 21:06:15.463552946 +0000 UTC m=+512.005715582"
Sep 15 21:06:26.436426 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:26.436389    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7b6486456d-h4hnf"
Sep 15 21:06:26.436890 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:26.436538    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-84f9876df9-z9cj7"
Sep 15 21:06:26.496320 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:26.496285    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84f9876df9-z9cj7"]
Sep 15 21:06:26.496534 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:26.496478    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-84f9876df9-z9cj7" podUID="e4686b9b-c253-4dcd-bb87-35a1924cd523" containerName="manager" containerID="cri-o://7dd9f6e48b97c33ee6af2a595073aea4efa3b8b40b9b9d2581467705c8ef076e" gracePeriod=30
Sep 15 21:06:26.734980 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:26.734951    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84f9876df9-z9cj7"
Sep 15 21:06:26.886870 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:26.886824    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6cc9l\" (UniqueName: \"kubernetes.io/projected/e4686b9b-c253-4dcd-bb87-35a1924cd523-kube-api-access-6cc9l\") pod \"e4686b9b-c253-4dcd-bb87-35a1924cd523\" (UID: \"e4686b9b-c253-4dcd-bb87-35a1924cd523\") "
Sep 15 21:06:26.886870 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:26.886870    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4686b9b-c253-4dcd-bb87-35a1924cd523-metrics-certs\") pod \"e4686b9b-c253-4dcd-bb87-35a1924cd523\" (UID: \"e4686b9b-c253-4dcd-bb87-35a1924cd523\") "
Sep 15 21:06:26.887132 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:26.886899    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4686b9b-c253-4dcd-bb87-35a1924cd523-webhook-cert\") pod \"e4686b9b-c253-4dcd-bb87-35a1924cd523\" (UID: \"e4686b9b-c253-4dcd-bb87-35a1924cd523\") "
Sep 15 21:06:26.889317 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:26.889279    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4686b9b-c253-4dcd-bb87-35a1924cd523-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e4686b9b-c253-4dcd-bb87-35a1924cd523" (UID: "e4686b9b-c253-4dcd-bb87-35a1924cd523"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:06:26.889317 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:26.889298    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4686b9b-c253-4dcd-bb87-35a1924cd523-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e4686b9b-c253-4dcd-bb87-35a1924cd523" (UID: "e4686b9b-c253-4dcd-bb87-35a1924cd523"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:06:26.889469 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:26.889396    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4686b9b-c253-4dcd-bb87-35a1924cd523-kube-api-access-6cc9l" (OuterVolumeSpecName: "kube-api-access-6cc9l") pod "e4686b9b-c253-4dcd-bb87-35a1924cd523" (UID: "e4686b9b-c253-4dcd-bb87-35a1924cd523"). InnerVolumeSpecName "kube-api-access-6cc9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:06:26.988613 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:26.988525    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cc9l\" (UniqueName: \"kubernetes.io/projected/e4686b9b-c253-4dcd-bb87-35a1924cd523-kube-api-access-6cc9l\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:06:26.988613 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:26.988563    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4686b9b-c253-4dcd-bb87-35a1924cd523-metrics-certs\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:06:26.988613 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:26.988574    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4686b9b-c253-4dcd-bb87-35a1924cd523-webhook-cert\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:06:27.484756 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:27.484718    2564 generic.go:356] "Generic (PLEG): container finished" podID="e4686b9b-c253-4dcd-bb87-35a1924cd523" containerID="7dd9f6e48b97c33ee6af2a595073aea4efa3b8b40b9b9d2581467705c8ef076e" exitCode=0
Sep 15 21:06:27.485159 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:27.484770    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84f9876df9-z9cj7"
Sep 15 21:06:27.485159 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:27.484778    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84f9876df9-z9cj7" event={"ID":"e4686b9b-c253-4dcd-bb87-35a1924cd523","Type":"ContainerDied","Data":"7dd9f6e48b97c33ee6af2a595073aea4efa3b8b40b9b9d2581467705c8ef076e"}
Sep 15 21:06:27.485159 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:27.484835    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84f9876df9-z9cj7" event={"ID":"e4686b9b-c253-4dcd-bb87-35a1924cd523","Type":"ContainerDied","Data":"7536766bcd9666828c2cb9753e371f71028fa15f833462f4d854083e434c770e"}
Sep 15 21:06:27.485159 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:27.484864    2564 scope.go:117] "RemoveContainer" containerID="7dd9f6e48b97c33ee6af2a595073aea4efa3b8b40b9b9d2581467705c8ef076e"
Sep 15 21:06:27.494218 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:27.494201    2564 scope.go:117] "RemoveContainer" containerID="7dd9f6e48b97c33ee6af2a595073aea4efa3b8b40b9b9d2581467705c8ef076e"
Sep 15 21:06:27.494530 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:06:27.494505    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dd9f6e48b97c33ee6af2a595073aea4efa3b8b40b9b9d2581467705c8ef076e\": container with ID starting with 7dd9f6e48b97c33ee6af2a595073aea4efa3b8b40b9b9d2581467705c8ef076e not found: ID does not exist" containerID="7dd9f6e48b97c33ee6af2a595073aea4efa3b8b40b9b9d2581467705c8ef076e"
Sep 15 21:06:27.494629 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:27.494537    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dd9f6e48b97c33ee6af2a595073aea4efa3b8b40b9b9d2581467705c8ef076e"} err="failed to get container status \"7dd9f6e48b97c33ee6af2a595073aea4efa3b8b40b9b9d2581467705c8ef076e\": rpc error: code = NotFound desc = could not find container \"7dd9f6e48b97c33ee6af2a595073aea4efa3b8b40b9b9d2581467705c8ef076e\": container with ID starting with 7dd9f6e48b97c33ee6af2a595073aea4efa3b8b40b9b9d2581467705c8ef076e not found: ID does not exist"
Sep 15 21:06:27.506572 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:27.506548    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84f9876df9-z9cj7"]
Sep 15 21:06:27.516541 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:27.516512    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-84f9876df9-z9cj7"]
Sep 15 21:06:27.972315 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:27.972278    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4686b9b-c253-4dcd-bb87-35a1924cd523" path="/var/lib/kubelet/pods/e4686b9b-c253-4dcd-bb87-35a1924cd523/volumes"
Sep 15 21:06:45.799078 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:45.799037    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:06:45.896604 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:45.896555    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:06:47.243850 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.243809    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:06:47.342556 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.342518    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:06:47.466010 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.465974    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:06:47.505399 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.505324    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"]
Sep 15 21:06:47.505693 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.505680    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4686b9b-c253-4dcd-bb87-35a1924cd523" containerName="manager"
Sep 15 21:06:47.505745 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.505695    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4686b9b-c253-4dcd-bb87-35a1924cd523" containerName="manager"
Sep 15 21:06:47.505783 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.505752    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4686b9b-c253-4dcd-bb87-35a1924cd523" containerName="manager"
Sep 15 21:06:47.508524 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.508506    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"
Sep 15 21:06:47.512385 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.512360    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 15 21:06:47.512385 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.512378    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8hfhw\""
Sep 15 21:06:47.512599 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.512357    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 15 21:06:47.518478 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.518455    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"]
Sep 15 21:06:47.560324 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.560286    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcddd559-66df-4dc4-9025-7de984f39336-maas-api-tls\") pod \"maas-api-646ff496d6-b6hvt\" (UID: \"fcddd559-66df-4dc4-9025-7de984f39336\") " pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"
Sep 15 21:06:47.560539 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.560372    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84t87\" (UniqueName: \"kubernetes.io/projected/fcddd559-66df-4dc4-9025-7de984f39336-kube-api-access-84t87\") pod \"maas-api-646ff496d6-b6hvt\" (UID: \"fcddd559-66df-4dc4-9025-7de984f39336\") " pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"
Sep 15 21:06:47.560539 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.560397    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcddd559-66df-4dc4-9025-7de984f39336-openshift-service-ca\") pod \"maas-api-646ff496d6-b6hvt\" (UID: \"fcddd559-66df-4dc4-9025-7de984f39336\") " pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"
Sep 15 21:06:47.560539 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.560462    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcddd559-66df-4dc4-9025-7de984f39336-metrics-certs\") pod \"maas-api-646ff496d6-b6hvt\" (UID: \"fcddd559-66df-4dc4-9025-7de984f39336\") " pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"
Sep 15 21:06:47.661026 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.660976    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84t87\" (UniqueName: \"kubernetes.io/projected/fcddd559-66df-4dc4-9025-7de984f39336-kube-api-access-84t87\") pod \"maas-api-646ff496d6-b6hvt\" (UID: \"fcddd559-66df-4dc4-9025-7de984f39336\") " pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"
Sep 15 21:06:47.661026 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.661024    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcddd559-66df-4dc4-9025-7de984f39336-openshift-service-ca\") pod \"maas-api-646ff496d6-b6hvt\" (UID: \"fcddd559-66df-4dc4-9025-7de984f39336\") " pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"
Sep 15 21:06:47.661283 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.661068    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcddd559-66df-4dc4-9025-7de984f39336-metrics-certs\") pod \"maas-api-646ff496d6-b6hvt\" (UID: \"fcddd559-66df-4dc4-9025-7de984f39336\") " pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"
Sep 15 21:06:47.661283 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.661093    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcddd559-66df-4dc4-9025-7de984f39336-maas-api-tls\") pod \"maas-api-646ff496d6-b6hvt\" (UID: \"fcddd559-66df-4dc4-9025-7de984f39336\") " pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"
Sep 15 21:06:47.661818 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.661794    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcddd559-66df-4dc4-9025-7de984f39336-openshift-service-ca\") pod \"maas-api-646ff496d6-b6hvt\" (UID: \"fcddd559-66df-4dc4-9025-7de984f39336\") " pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"
Sep 15 21:06:47.663766 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.663746    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcddd559-66df-4dc4-9025-7de984f39336-maas-api-tls\") pod \"maas-api-646ff496d6-b6hvt\" (UID: \"fcddd559-66df-4dc4-9025-7de984f39336\") " pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"
Sep 15 21:06:47.663865 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.663810    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcddd559-66df-4dc4-9025-7de984f39336-metrics-certs\") pod \"maas-api-646ff496d6-b6hvt\" (UID: \"fcddd559-66df-4dc4-9025-7de984f39336\") " pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"
Sep 15 21:06:47.669533 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.669478    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84t87\" (UniqueName: \"kubernetes.io/projected/fcddd559-66df-4dc4-9025-7de984f39336-kube-api-access-84t87\") pod \"maas-api-646ff496d6-b6hvt\" (UID: \"fcddd559-66df-4dc4-9025-7de984f39336\") " pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"
Sep 15 21:06:47.680613 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.680587    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:06:47.819736 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.819654    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"
Sep 15 21:06:47.919180 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:47.919148    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:06:48.022660 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:48.022624    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-cwq7f"]
Sep 15 21:06:48.024920 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:48.024895    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:06:48.037763 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:48.037718    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-cwq7f"]
Sep 15 21:06:48.037914 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:48.037873    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cwq7f"
Sep 15 21:06:48.041173 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:48.041148    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 15 21:06:48.041312 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:48.041212    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-n8k8b\""
Sep 15 21:06:48.121743 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:48.121698    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"]
Sep 15 21:06:48.126143 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:06:48.126105    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcddd559_66df_4dc4_9025_7de984f39336.slice/crio-462a6c45563271aef561edc4a2f270f97d231de660bd51f882468be7638f6dc9 WatchSource:0}: Error finding container 462a6c45563271aef561edc4a2f270f97d231de660bd51f882468be7638f6dc9: Status 404 returned error can't find the container with id 462a6c45563271aef561edc4a2f270f97d231de660bd51f882468be7638f6dc9
Sep 15 21:06:48.164657 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:48.164588    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb436781-9ba7-4346-b984-b40c1a82edc6-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-cwq7f\" (UID: \"bb436781-9ba7-4346-b984-b40c1a82edc6\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cwq7f"
Sep 15 21:06:48.165156 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:48.164762    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8hcb\" (UniqueName: \"kubernetes.io/projected/bb436781-9ba7-4346-b984-b40c1a82edc6-kube-api-access-g8hcb\") pod \"payload-pre-processing-59fbc47bf8-cwq7f\" (UID: \"bb436781-9ba7-4346-b984-b40c1a82edc6\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cwq7f"
Sep 15 21:06:48.265246 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:48.265205    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb436781-9ba7-4346-b984-b40c1a82edc6-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-cwq7f\" (UID: \"bb436781-9ba7-4346-b984-b40c1a82edc6\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cwq7f"
Sep 15 21:06:48.265736 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:48.265281    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8hcb\" (UniqueName: \"kubernetes.io/projected/bb436781-9ba7-4346-b984-b40c1a82edc6-kube-api-access-g8hcb\") pod \"payload-pre-processing-59fbc47bf8-cwq7f\" (UID: \"bb436781-9ba7-4346-b984-b40c1a82edc6\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cwq7f"
Sep 15 21:06:48.266099 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:48.266072    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb436781-9ba7-4346-b984-b40c1a82edc6-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-cwq7f\" (UID: \"bb436781-9ba7-4346-b984-b40c1a82edc6\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cwq7f"
Sep 15 21:06:48.274203 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:48.274170    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8hcb\" (UniqueName: \"kubernetes.io/projected/bb436781-9ba7-4346-b984-b40c1a82edc6-kube-api-access-g8hcb\") pod \"payload-pre-processing-59fbc47bf8-cwq7f\" (UID: \"bb436781-9ba7-4346-b984-b40c1a82edc6\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cwq7f"
Sep 15 21:06:48.349417 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:48.349386    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cwq7f"
Sep 15 21:06:48.581677 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:48.581645    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt" event={"ID":"fcddd559-66df-4dc4-9025-7de984f39336","Type":"ContainerStarted","Data":"462a6c45563271aef561edc4a2f270f97d231de660bd51f882468be7638f6dc9"}
Sep 15 21:06:48.622390 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:48.622354    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-cwq7f"]
Sep 15 21:06:48.626590 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:06:48.626562    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb436781_9ba7_4346_b984_b40c1a82edc6.slice/crio-941c83ccb16f607d3c75f81da6ec46406eff8aa38b9f95085bb6d302d0f90706 WatchSource:0}: Error finding container 941c83ccb16f607d3c75f81da6ec46406eff8aa38b9f95085bb6d302d0f90706: Status 404 returned error can't find the container with id 941c83ccb16f607d3c75f81da6ec46406eff8aa38b9f95085bb6d302d0f90706
Sep 15 21:06:49.589519 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:49.589464    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cwq7f" event={"ID":"bb436781-9ba7-4346-b984-b40c1a82edc6","Type":"ContainerStarted","Data":"941c83ccb16f607d3c75f81da6ec46406eff8aa38b9f95085bb6d302d0f90706"}
Sep 15 21:06:52.606407 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:52.606373    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt" event={"ID":"fcddd559-66df-4dc4-9025-7de984f39336","Type":"ContainerStarted","Data":"5c13f8ab877b598dc4ce92a1fc404e1d694bcf1bff4103c86d6517d6f6b1ccd3"}
Sep 15 21:06:52.606926 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:52.606515    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"
Sep 15 21:06:52.608202 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:52.608174    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cwq7f" event={"ID":"bb436781-9ba7-4346-b984-b40c1a82edc6","Type":"ContainerStarted","Data":"7be90f48a6f9925c9824306a2768eba3cdc607cd24df9016772caa3db619006a"}
Sep 15 21:06:52.608318 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:52.608245    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cwq7f"
Sep 15 21:06:52.623964 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:52.623907    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt" podStartSLOduration=2.163990871 podStartE2EDuration="5.623890808s" totalImagesPullingTime="3.459899937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:06:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:06:52.623632529 +0000 UTC m=+549.165795184" watchObservedRunningTime="2026-09-15 21:06:52.623890808 +0000 UTC m=+549.166053442"
Sep 15 21:06:52.639850 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:52.639798    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cwq7f" podStartSLOduration=1.680293078 podStartE2EDuration="4.639781044s" totalImagesPullingTime="2.959487966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:06:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:06:52.639309433 +0000 UTC m=+549.181472085" watchObservedRunningTime="2026-09-15 21:06:52.639781044 +0000 UTC m=+549.181943721"
Sep 15 21:06:56.992839 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:56.992785    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:06:57.044221 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:57.044187    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:06:58.623902 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:58.623860    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"
Sep 15 21:06:59.648469 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:06:59.648433    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:07:00.034766 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:00.034665    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:07:03.615517 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:03.615456    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cwq7f"
Sep 15 21:07:07.392638 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.392594    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"]
Sep 15 21:07:07.396085 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.396067    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:07.400502 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.400464    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 15 21:07:07.400645 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.400466    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 15 21:07:07.400645 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.400465    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 15 21:07:07.400645 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.400545    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-b2fk7\""
Sep 15 21:07:07.405542 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.405517    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"]
Sep 15 21:07:07.426046 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.426016    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:07:07.537787 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.537753    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5791690e-4c5d-4aff-8f8c-e437e809121f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl\" (UID: \"5791690e-4c5d-4aff-8f8c-e437e809121f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:07.537787 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.537788    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5791690e-4c5d-4aff-8f8c-e437e809121f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl\" (UID: \"5791690e-4c5d-4aff-8f8c-e437e809121f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:07.538005 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.537820    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5791690e-4c5d-4aff-8f8c-e437e809121f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl\" (UID: \"5791690e-4c5d-4aff-8f8c-e437e809121f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:07.538005 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.537886    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5791690e-4c5d-4aff-8f8c-e437e809121f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl\" (UID: \"5791690e-4c5d-4aff-8f8c-e437e809121f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:07.538005 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.537945    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptgrs\" (UniqueName: \"kubernetes.io/projected/5791690e-4c5d-4aff-8f8c-e437e809121f-kube-api-access-ptgrs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl\" (UID: \"5791690e-4c5d-4aff-8f8c-e437e809121f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:07.538103 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.538040    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5791690e-4c5d-4aff-8f8c-e437e809121f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl\" (UID: \"5791690e-4c5d-4aff-8f8c-e437e809121f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:07.639357 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.639320    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5791690e-4c5d-4aff-8f8c-e437e809121f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl\" (UID: \"5791690e-4c5d-4aff-8f8c-e437e809121f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:07.639357 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.639360    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5791690e-4c5d-4aff-8f8c-e437e809121f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl\" (UID: \"5791690e-4c5d-4aff-8f8c-e437e809121f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:07.639621 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.639393    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptgrs\" (UniqueName: \"kubernetes.io/projected/5791690e-4c5d-4aff-8f8c-e437e809121f-kube-api-access-ptgrs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl\" (UID: \"5791690e-4c5d-4aff-8f8c-e437e809121f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:07.639621 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.639454    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5791690e-4c5d-4aff-8f8c-e437e809121f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl\" (UID: \"5791690e-4c5d-4aff-8f8c-e437e809121f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:07.639621 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.639525    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5791690e-4c5d-4aff-8f8c-e437e809121f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl\" (UID: \"5791690e-4c5d-4aff-8f8c-e437e809121f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:07.639621 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.639546    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5791690e-4c5d-4aff-8f8c-e437e809121f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl\" (UID: \"5791690e-4c5d-4aff-8f8c-e437e809121f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:07.639863 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.639828    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5791690e-4c5d-4aff-8f8c-e437e809121f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl\" (UID: \"5791690e-4c5d-4aff-8f8c-e437e809121f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:07.639863 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.639858    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5791690e-4c5d-4aff-8f8c-e437e809121f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl\" (UID: \"5791690e-4c5d-4aff-8f8c-e437e809121f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:07.639971 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.639924    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5791690e-4c5d-4aff-8f8c-e437e809121f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl\" (UID: \"5791690e-4c5d-4aff-8f8c-e437e809121f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:07.641957 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.641927    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5791690e-4c5d-4aff-8f8c-e437e809121f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl\" (UID: \"5791690e-4c5d-4aff-8f8c-e437e809121f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:07.642174 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.642155    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5791690e-4c5d-4aff-8f8c-e437e809121f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl\" (UID: \"5791690e-4c5d-4aff-8f8c-e437e809121f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:07.649109 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.649047    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptgrs\" (UniqueName: \"kubernetes.io/projected/5791690e-4c5d-4aff-8f8c-e437e809121f-kube-api-access-ptgrs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl\" (UID: \"5791690e-4c5d-4aff-8f8c-e437e809121f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:07.706959 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.706920    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:07.981559 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:07.981534    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"]
Sep 15 21:07:07.983375 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:07:07.983340    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5791690e_4c5d_4aff_8f8c_e437e809121f.slice/crio-6962f9b4964c7d85e46115aaa0955d4b7cff2ff0a3a1b544839cb4360c904918 WatchSource:0}: Error finding container 6962f9b4964c7d85e46115aaa0955d4b7cff2ff0a3a1b544839cb4360c904918: Status 404 returned error can't find the container with id 6962f9b4964c7d85e46115aaa0955d4b7cff2ff0a3a1b544839cb4360c904918
Sep 15 21:07:08.682018 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:08.681981    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl" event={"ID":"5791690e-4c5d-4aff-8f8c-e437e809121f","Type":"ContainerStarted","Data":"6962f9b4964c7d85e46115aaa0955d4b7cff2ff0a3a1b544839cb4360c904918"}
Sep 15 21:07:09.689438 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:09.689404    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl" event={"ID":"5791690e-4c5d-4aff-8f8c-e437e809121f","Type":"ContainerStarted","Data":"06c97ed1a2ef6ab38395c40e5b98d2102f0ffcb17753a6819c049083a63c1f5c"}
Sep 15 21:07:09.707369 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:09.707313    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl" podStartSLOduration=1.5877933450000001 podStartE2EDuration="2.707294406s" totalImagesPullingTime="1.119501061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:07:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:07:09.705777079 +0000 UTC m=+566.247939736" watchObservedRunningTime="2026-09-15 21:07:09.707294406 +0000 UTC m=+566.249457065"
Sep 15 21:07:17.707285 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:17.707238    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:17.707846 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:17.707312    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:17.720046 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:17.720018    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:17.739093 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:17.739063    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl"
Sep 15 21:07:30.589249 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.589215    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"]
Sep 15 21:07:30.594709 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.594684    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:30.597525 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.597483    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 15 21:07:30.603812 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.603784    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"]
Sep 15 21:07:30.627589 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.627555    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6440c331-fa9c-4264-b61b-c48c30bd2492-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t44sl\" (UID: \"6440c331-fa9c-4264-b61b-c48c30bd2492\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:30.627747 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.627693    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6440c331-fa9c-4264-b61b-c48c30bd2492-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t44sl\" (UID: \"6440c331-fa9c-4264-b61b-c48c30bd2492\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:30.627747 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.627731    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6440c331-fa9c-4264-b61b-c48c30bd2492-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t44sl\" (UID: \"6440c331-fa9c-4264-b61b-c48c30bd2492\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:30.627861 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.627755    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6440c331-fa9c-4264-b61b-c48c30bd2492-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t44sl\" (UID: \"6440c331-fa9c-4264-b61b-c48c30bd2492\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:30.627861 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.627770    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6440c331-fa9c-4264-b61b-c48c30bd2492-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t44sl\" (UID: \"6440c331-fa9c-4264-b61b-c48c30bd2492\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:30.627861 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.627789    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc2qr\" (UniqueName: \"kubernetes.io/projected/6440c331-fa9c-4264-b61b-c48c30bd2492-kube-api-access-xc2qr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t44sl\" (UID: \"6440c331-fa9c-4264-b61b-c48c30bd2492\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:30.629420 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.629394    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:07:30.728415 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.728380    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6440c331-fa9c-4264-b61b-c48c30bd2492-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t44sl\" (UID: \"6440c331-fa9c-4264-b61b-c48c30bd2492\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:30.728415 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.728414    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6440c331-fa9c-4264-b61b-c48c30bd2492-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t44sl\" (UID: \"6440c331-fa9c-4264-b61b-c48c30bd2492\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:30.728712 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.728436    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6440c331-fa9c-4264-b61b-c48c30bd2492-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t44sl\" (UID: \"6440c331-fa9c-4264-b61b-c48c30bd2492\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:30.728712 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.728575    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6440c331-fa9c-4264-b61b-c48c30bd2492-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t44sl\" (UID: \"6440c331-fa9c-4264-b61b-c48c30bd2492\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:30.728712 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.728635    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc2qr\" (UniqueName: \"kubernetes.io/projected/6440c331-fa9c-4264-b61b-c48c30bd2492-kube-api-access-xc2qr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t44sl\" (UID: \"6440c331-fa9c-4264-b61b-c48c30bd2492\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:30.728712 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.728674    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6440c331-fa9c-4264-b61b-c48c30bd2492-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t44sl\" (UID: \"6440c331-fa9c-4264-b61b-c48c30bd2492\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:30.728901 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.728871    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6440c331-fa9c-4264-b61b-c48c30bd2492-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t44sl\" (UID: \"6440c331-fa9c-4264-b61b-c48c30bd2492\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:30.728940 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.728898    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6440c331-fa9c-4264-b61b-c48c30bd2492-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t44sl\" (UID: \"6440c331-fa9c-4264-b61b-c48c30bd2492\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:30.728982 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.728942    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6440c331-fa9c-4264-b61b-c48c30bd2492-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t44sl\" (UID: \"6440c331-fa9c-4264-b61b-c48c30bd2492\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:30.731009 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.730988    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6440c331-fa9c-4264-b61b-c48c30bd2492-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t44sl\" (UID: \"6440c331-fa9c-4264-b61b-c48c30bd2492\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:30.731172 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.731154    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6440c331-fa9c-4264-b61b-c48c30bd2492-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t44sl\" (UID: \"6440c331-fa9c-4264-b61b-c48c30bd2492\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:30.737288 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.737261    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc2qr\" (UniqueName: \"kubernetes.io/projected/6440c331-fa9c-4264-b61b-c48c30bd2492-kube-api-access-xc2qr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t44sl\" (UID: \"6440c331-fa9c-4264-b61b-c48c30bd2492\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:30.907405 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:30.907356    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:31.182074 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:31.182044    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"]
Sep 15 21:07:31.183782 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:07:31.183751    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6440c331_fa9c_4264_b61b_c48c30bd2492.slice/crio-eba3ee9bdee59bf4e0dca3c683a117e49cbb4378b2351bb06edbbf3c202fe0ad WatchSource:0}: Error finding container eba3ee9bdee59bf4e0dca3c683a117e49cbb4378b2351bb06edbbf3c202fe0ad: Status 404 returned error can't find the container with id eba3ee9bdee59bf4e0dca3c683a117e49cbb4378b2351bb06edbbf3c202fe0ad
Sep 15 21:07:31.794656 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:31.794621    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl" event={"ID":"6440c331-fa9c-4264-b61b-c48c30bd2492","Type":"ContainerStarted","Data":"5061386ecfc0a93642a7f7406b502dbb521e8ca30fe2ed0ffec033c4d1a1dfbd"}
Sep 15 21:07:31.795046 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:31.794674    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl" event={"ID":"6440c331-fa9c-4264-b61b-c48c30bd2492","Type":"ContainerStarted","Data":"eba3ee9bdee59bf4e0dca3c683a117e49cbb4378b2351bb06edbbf3c202fe0ad"}
Sep 15 21:07:31.813093 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:31.813040    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl" podStartSLOduration=1.632117459 podStartE2EDuration="1.813019339s" totalImagesPullingTime="180.90188ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:07:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:07:31.812554622 +0000 UTC m=+588.354717283" watchObservedRunningTime="2026-09-15 21:07:31.813019339 +0000 UTC m=+588.355181974"
Sep 15 21:07:38.014119 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:38.014085    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:07:38.075350 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:38.075317    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:07:40.908438 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:40.908399    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:40.908928 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:40.908471    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:40.921125 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:40.921100    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:41.854928 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:41.854897    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t44sl"
Sep 15 21:07:43.910618 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:43.910592    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-7t46m_c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf/console-operator/1.log"
Sep 15 21:07:43.911036 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:43.910627    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-7t46m_c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf/console-operator/1.log"
Sep 15 21:07:43.913934 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:43.913916    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4c7k_fc205950-caf0-4c47-84ce-a8fc54a1afb2/ovn-acl-logging/0.log"
Sep 15 21:07:43.914063 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:43.913921    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4c7k_fc205950-caf0-4c47-84ce-a8fc54a1afb2/ovn-acl-logging/0.log"
Sep 15 21:07:47.909081 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:47.909044    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:07:58.501731 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.501696    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"]
Sep 15 21:07:58.504263 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.504243    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:07:58.506999 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.506976    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 15 21:07:58.517334 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.517297    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"]
Sep 15 21:07:58.583542 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.583477    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f13a0a9f-8abf-468a-b8b4-77a1057747f1-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-j4s6q\" (UID: \"f13a0a9f-8abf-468a-b8b4-77a1057747f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:07:58.583737 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.583565    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f13a0a9f-8abf-468a-b8b4-77a1057747f1-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-j4s6q\" (UID: \"f13a0a9f-8abf-468a-b8b4-77a1057747f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:07:58.583737 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.583690    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f13a0a9f-8abf-468a-b8b4-77a1057747f1-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-j4s6q\" (UID: \"f13a0a9f-8abf-468a-b8b4-77a1057747f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:07:58.583737 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.583731    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f13a0a9f-8abf-468a-b8b4-77a1057747f1-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-j4s6q\" (UID: \"f13a0a9f-8abf-468a-b8b4-77a1057747f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:07:58.583905 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.583763    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f13a0a9f-8abf-468a-b8b4-77a1057747f1-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-j4s6q\" (UID: \"f13a0a9f-8abf-468a-b8b4-77a1057747f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:07:58.583905 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.583790    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2lm4\" (UniqueName: \"kubernetes.io/projected/f13a0a9f-8abf-468a-b8b4-77a1057747f1-kube-api-access-t2lm4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-j4s6q\" (UID: \"f13a0a9f-8abf-468a-b8b4-77a1057747f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:07:58.638641 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.638596    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:07:58.684708 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.684671    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f13a0a9f-8abf-468a-b8b4-77a1057747f1-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-j4s6q\" (UID: \"f13a0a9f-8abf-468a-b8b4-77a1057747f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:07:58.684708 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.684713    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2lm4\" (UniqueName: \"kubernetes.io/projected/f13a0a9f-8abf-468a-b8b4-77a1057747f1-kube-api-access-t2lm4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-j4s6q\" (UID: \"f13a0a9f-8abf-468a-b8b4-77a1057747f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:07:58.684921 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.684741    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f13a0a9f-8abf-468a-b8b4-77a1057747f1-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-j4s6q\" (UID: \"f13a0a9f-8abf-468a-b8b4-77a1057747f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:07:58.684921 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.684778    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f13a0a9f-8abf-468a-b8b4-77a1057747f1-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-j4s6q\" (UID: \"f13a0a9f-8abf-468a-b8b4-77a1057747f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:07:58.684921 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.684839    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f13a0a9f-8abf-468a-b8b4-77a1057747f1-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-j4s6q\" (UID: \"f13a0a9f-8abf-468a-b8b4-77a1057747f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:07:58.684921 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.684862    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f13a0a9f-8abf-468a-b8b4-77a1057747f1-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-j4s6q\" (UID: \"f13a0a9f-8abf-468a-b8b4-77a1057747f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:07:58.685190 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.685166    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f13a0a9f-8abf-468a-b8b4-77a1057747f1-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-j4s6q\" (UID: \"f13a0a9f-8abf-468a-b8b4-77a1057747f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:07:58.685262 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.685185    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f13a0a9f-8abf-468a-b8b4-77a1057747f1-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-j4s6q\" (UID: \"f13a0a9f-8abf-468a-b8b4-77a1057747f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:07:58.685323 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.685256    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f13a0a9f-8abf-468a-b8b4-77a1057747f1-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-j4s6q\" (UID: \"f13a0a9f-8abf-468a-b8b4-77a1057747f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:07:58.687601 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.687575    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f13a0a9f-8abf-468a-b8b4-77a1057747f1-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-j4s6q\" (UID: \"f13a0a9f-8abf-468a-b8b4-77a1057747f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:07:58.687749 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.687709    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f13a0a9f-8abf-468a-b8b4-77a1057747f1-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-j4s6q\" (UID: \"f13a0a9f-8abf-468a-b8b4-77a1057747f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:07:58.693922 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.693900    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2lm4\" (UniqueName: \"kubernetes.io/projected/f13a0a9f-8abf-468a-b8b4-77a1057747f1-kube-api-access-t2lm4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-j4s6q\" (UID: \"f13a0a9f-8abf-468a-b8b4-77a1057747f1\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:07:58.815438 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:58.815337    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:07:59.088078 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:59.088040    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"]
Sep 15 21:07:59.089502 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:07:59.089465    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf13a0a9f_8abf_468a_b8b4_77a1057747f1.slice/crio-5c237b05c059deaefd627db5d4b330520d88703644dd0a32e5a39b90645ff6a4 WatchSource:0}: Error finding container 5c237b05c059deaefd627db5d4b330520d88703644dd0a32e5a39b90645ff6a4: Status 404 returned error can't find the container with id 5c237b05c059deaefd627db5d4b330520d88703644dd0a32e5a39b90645ff6a4
Sep 15 21:07:59.931027 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:59.930992    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q" event={"ID":"f13a0a9f-8abf-468a-b8b4-77a1057747f1","Type":"ContainerStarted","Data":"9365ef004631297e2f8ab207e560477a5819a42ba3010cbc80c20fccb294ae50"}
Sep 15 21:07:59.931402 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:59.931044    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q" event={"ID":"f13a0a9f-8abf-468a-b8b4-77a1057747f1","Type":"ContainerStarted","Data":"5c237b05c059deaefd627db5d4b330520d88703644dd0a32e5a39b90645ff6a4"}
Sep 15 21:07:59.951135 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:07:59.951083    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q" podStartSLOduration=1.7722768530000002 podStartE2EDuration="1.951066017s" totalImagesPullingTime="178.789164ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:07:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:07:59.949945992 +0000 UTC m=+616.492108629" watchObservedRunningTime="2026-09-15 21:07:59.951066017 +0000 UTC m=+616.493228651"
Sep 15 21:08:08.816460 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:08.816379    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:08:08.816460 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:08.816441    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:08:08.829144 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:08.829120    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:08:08.984891 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:08.984859    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-j4s6q"
Sep 15 21:08:09.944710 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:09.944671    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:08:11.930986 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:11.930941    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:08:20.536054 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:20.536013    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:08:25.987228 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:25.987199    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"]
Sep 15 21:08:25.989599 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:25.989576    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:25.992741 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:25.992718    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 15 21:08:26.007110 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:26.007075    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"]
Sep 15 21:08:26.029016 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:26.028979    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4de8b482-cceb-4bec-a1be-b550eba701b0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gs4d4\" (UID: \"4de8b482-cceb-4bec-a1be-b550eba701b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:26.029184 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:26.029033    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4de8b482-cceb-4bec-a1be-b550eba701b0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gs4d4\" (UID: \"4de8b482-cceb-4bec-a1be-b550eba701b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:26.029184 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:26.029057    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4de8b482-cceb-4bec-a1be-b550eba701b0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gs4d4\" (UID: \"4de8b482-cceb-4bec-a1be-b550eba701b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:26.029184 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:26.029075    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4de8b482-cceb-4bec-a1be-b550eba701b0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gs4d4\" (UID: \"4de8b482-cceb-4bec-a1be-b550eba701b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:26.029345 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:26.029234    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4de8b482-cceb-4bec-a1be-b550eba701b0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gs4d4\" (UID: \"4de8b482-cceb-4bec-a1be-b550eba701b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:26.029345 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:26.029272    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2rzp\" (UniqueName: \"kubernetes.io/projected/4de8b482-cceb-4bec-a1be-b550eba701b0-kube-api-access-x2rzp\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gs4d4\" (UID: \"4de8b482-cceb-4bec-a1be-b550eba701b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:26.076434 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:26.076397    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:08:26.130463 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:26.130420    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4de8b482-cceb-4bec-a1be-b550eba701b0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gs4d4\" (UID: \"4de8b482-cceb-4bec-a1be-b550eba701b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:26.130647 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:26.130469    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4de8b482-cceb-4bec-a1be-b550eba701b0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gs4d4\" (UID: \"4de8b482-cceb-4bec-a1be-b550eba701b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:26.130647 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:26.130524    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4de8b482-cceb-4bec-a1be-b550eba701b0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gs4d4\" (UID: \"4de8b482-cceb-4bec-a1be-b550eba701b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:26.130647 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:26.130585    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4de8b482-cceb-4bec-a1be-b550eba701b0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gs4d4\" (UID: \"4de8b482-cceb-4bec-a1be-b550eba701b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:26.130647 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:26.130612    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2rzp\" (UniqueName: \"kubernetes.io/projected/4de8b482-cceb-4bec-a1be-b550eba701b0-kube-api-access-x2rzp\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gs4d4\" (UID: \"4de8b482-cceb-4bec-a1be-b550eba701b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:26.130841 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:26.130649    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4de8b482-cceb-4bec-a1be-b550eba701b0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gs4d4\" (UID: \"4de8b482-cceb-4bec-a1be-b550eba701b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:26.131007 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:26.130980    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4de8b482-cceb-4bec-a1be-b550eba701b0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gs4d4\" (UID: \"4de8b482-cceb-4bec-a1be-b550eba701b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:26.131007 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:26.130996    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4de8b482-cceb-4bec-a1be-b550eba701b0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gs4d4\" (UID: \"4de8b482-cceb-4bec-a1be-b550eba701b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:26.131391 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:26.131047    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4de8b482-cceb-4bec-a1be-b550eba701b0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gs4d4\" (UID: \"4de8b482-cceb-4bec-a1be-b550eba701b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:26.133105 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:26.133083    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4de8b482-cceb-4bec-a1be-b550eba701b0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gs4d4\" (UID: \"4de8b482-cceb-4bec-a1be-b550eba701b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:26.133358 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:26.133342    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4de8b482-cceb-4bec-a1be-b550eba701b0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gs4d4\" (UID: \"4de8b482-cceb-4bec-a1be-b550eba701b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:26.141889 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:26.141862    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2rzp\" (UniqueName: \"kubernetes.io/projected/4de8b482-cceb-4bec-a1be-b550eba701b0-kube-api-access-x2rzp\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gs4d4\" (UID: \"4de8b482-cceb-4bec-a1be-b550eba701b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:26.302009 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:26.301910    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:26.575182 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:26.575153    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"]
Sep 15 21:08:26.577266 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:08:26.577238    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4de8b482_cceb_4bec_a1be_b550eba701b0.slice/crio-4b95cdbbb80e616c0761c3d0ef682d73eac525ad28e4e9d9a75d9d2a8cec48c5 WatchSource:0}: Error finding container 4b95cdbbb80e616c0761c3d0ef682d73eac525ad28e4e9d9a75d9d2a8cec48c5: Status 404 returned error can't find the container with id 4b95cdbbb80e616c0761c3d0ef682d73eac525ad28e4e9d9a75d9d2a8cec48c5
Sep 15 21:08:26.578533 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:26.578515    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 21:08:27.058041 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:27.058001    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4" event={"ID":"4de8b482-cceb-4bec-a1be-b550eba701b0","Type":"ContainerStarted","Data":"fb4e4ab8217e180546c149f9bcc5aae7d4e384ae5f83c44b506b9ff51da7e289"}
Sep 15 21:08:27.058419 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:27.058065    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4" event={"ID":"4de8b482-cceb-4bec-a1be-b550eba701b0","Type":"ContainerStarted","Data":"4b95cdbbb80e616c0761c3d0ef682d73eac525ad28e4e9d9a75d9d2a8cec48c5"}
Sep 15 21:08:27.078864 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:27.078809    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4" podStartSLOduration=1.907662219 podStartE2EDuration="2.078791521s" totalImagesPullingTime="171.129302ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:08:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:08:27.07670829 +0000 UTC m=+643.618870925" watchObservedRunningTime="2026-09-15 21:08:27.078791521 +0000 UTC m=+643.620954156"
Sep 15 21:08:30.642304 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:30.642257    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:08:35.170778 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:35.170746    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:08:36.302038 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:36.301993    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:36.302424 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:36.302058    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:36.314594 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:36.314564    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:37.118222 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:37.118194    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gs4d4"
Sep 15 21:08:42.992638 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:08:42.992602    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:09:59.090657 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:09:59.090613    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:09:59.195764 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:09:59.195724    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:00.305831 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:00.305795    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:00.413189 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:00.413158    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:16.879817 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:16.879755    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:16.981272 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:16.981233    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:17.081774 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:17.081741    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:17.296539 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:17.296423    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:17.405136 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:17.405102    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:17.493382 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:17.493345    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:17.786430 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:17.786395    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:17.874580 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:17.874544    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:18.360508 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:18.360470    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:18.458747 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:18.458711    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:18.872083 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:18.872046    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:20.163285 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:20.163247    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:20.251873 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:20.251827    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:20.345424 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:20.345393    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:21.434608 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:21.434569    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"]
Sep 15 21:10:21.438399 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:21.438369    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"
Sep 15 21:10:21.443105 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:21.443073    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-dca193-serving-cert\""
Sep 15 21:10:21.461252 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:21.461221    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"]
Sep 15 21:10:21.512980 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:21.512936    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:21.571551 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:21.571511    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98b8aab3-7b1d-4723-b152-ec7f53357a2b-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-dca193-ffb957666-thstw\" (UID: \"98b8aab3-7b1d-4723-b152-ec7f53357a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"
Sep 15 21:10:21.571551 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:21.571555    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cnp6\" (UniqueName: \"kubernetes.io/projected/98b8aab3-7b1d-4723-b152-ec7f53357a2b-kube-api-access-2cnp6\") pod \"maas-api-e2e-worker-w4-dca193-ffb957666-thstw\" (UID: \"98b8aab3-7b1d-4723-b152-ec7f53357a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"
Sep 15 21:10:21.571766 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:21.571607    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98b8aab3-7b1d-4723-b152-ec7f53357a2b-metrics-certs\") pod \"maas-api-e2e-worker-w4-dca193-ffb957666-thstw\" (UID: \"98b8aab3-7b1d-4723-b152-ec7f53357a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"
Sep 15 21:10:21.571766 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:21.571646    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98b8aab3-7b1d-4723-b152-ec7f53357a2b-maas-api-tls\") pod \"maas-api-e2e-worker-w4-dca193-ffb957666-thstw\" (UID: \"98b8aab3-7b1d-4723-b152-ec7f53357a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"
Sep 15 21:10:21.593293 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:21.593263    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:21.654764 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:21.654732    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:21.673010 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:21.672957    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cnp6\" (UniqueName: \"kubernetes.io/projected/98b8aab3-7b1d-4723-b152-ec7f53357a2b-kube-api-access-2cnp6\") pod \"maas-api-e2e-worker-w4-dca193-ffb957666-thstw\" (UID: \"98b8aab3-7b1d-4723-b152-ec7f53357a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"
Sep 15 21:10:21.673181 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:21.673029    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98b8aab3-7b1d-4723-b152-ec7f53357a2b-metrics-certs\") pod \"maas-api-e2e-worker-w4-dca193-ffb957666-thstw\" (UID: \"98b8aab3-7b1d-4723-b152-ec7f53357a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"
Sep 15 21:10:21.673181 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:21.673068    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98b8aab3-7b1d-4723-b152-ec7f53357a2b-maas-api-tls\") pod \"maas-api-e2e-worker-w4-dca193-ffb957666-thstw\" (UID: \"98b8aab3-7b1d-4723-b152-ec7f53357a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"
Sep 15 21:10:21.673181 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:21.673161    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98b8aab3-7b1d-4723-b152-ec7f53357a2b-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-dca193-ffb957666-thstw\" (UID: \"98b8aab3-7b1d-4723-b152-ec7f53357a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"
Sep 15 21:10:21.673989 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:21.673898    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98b8aab3-7b1d-4723-b152-ec7f53357a2b-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-dca193-ffb957666-thstw\" (UID: \"98b8aab3-7b1d-4723-b152-ec7f53357a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"
Sep 15 21:10:21.675830 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:21.675805    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98b8aab3-7b1d-4723-b152-ec7f53357a2b-metrics-certs\") pod \"maas-api-e2e-worker-w4-dca193-ffb957666-thstw\" (UID: \"98b8aab3-7b1d-4723-b152-ec7f53357a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"
Sep 15 21:10:21.675975 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:21.675902    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98b8aab3-7b1d-4723-b152-ec7f53357a2b-maas-api-tls\") pod \"maas-api-e2e-worker-w4-dca193-ffb957666-thstw\" (UID: \"98b8aab3-7b1d-4723-b152-ec7f53357a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"
Sep 15 21:10:21.683157 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:21.683123    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cnp6\" (UniqueName: \"kubernetes.io/projected/98b8aab3-7b1d-4723-b152-ec7f53357a2b-kube-api-access-2cnp6\") pod \"maas-api-e2e-worker-w4-dca193-ffb957666-thstw\" (UID: \"98b8aab3-7b1d-4723-b152-ec7f53357a2b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"
Sep 15 21:10:21.750830 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:21.750737    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"
Sep 15 21:10:22.026310 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.026284    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"]
Sep 15 21:10:22.029129 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:10:22.029099    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98b8aab3_7b1d_4723_b152_ec7f53357a2b.slice/crio-3e48606358f303e54ca301d615ff5b6588412ba4f1bfec9ca71ec720bf8994f1 WatchSource:0}: Error finding container 3e48606358f303e54ca301d615ff5b6588412ba4f1bfec9ca71ec720bf8994f1: Status 404 returned error can't find the container with id 3e48606358f303e54ca301d615ff5b6588412ba4f1bfec9ca71ec720bf8994f1
Sep 15 21:10:22.224746 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.224706    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:22.325186 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.325086    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:22.344666 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.344628    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7"]
Sep 15 21:10:22.349428 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.349397    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7"
Sep 15 21:10:22.353250 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.353229    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-dca193-dockercfg-l4jdm\""
Sep 15 21:10:22.353580 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.353563    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-dca193\""
Sep 15 21:10:22.359649 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.359617    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7"]
Sep 15 21:10:22.481456 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.481416    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c11383a-a64c-49e0-9b91-110607ed70be-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7\" (UID: \"3c11383a-a64c-49e0-9b91-110607ed70be\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7"
Sep 15 21:10:22.481885 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.481577    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tggmb\" (UniqueName: \"kubernetes.io/projected/3c11383a-a64c-49e0-9b91-110607ed70be-kube-api-access-tggmb\") pod \"payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7\" (UID: \"3c11383a-a64c-49e0-9b91-110607ed70be\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7"
Sep 15 21:10:22.582277 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.582182    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tggmb\" (UniqueName: \"kubernetes.io/projected/3c11383a-a64c-49e0-9b91-110607ed70be-kube-api-access-tggmb\") pod \"payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7\" (UID: \"3c11383a-a64c-49e0-9b91-110607ed70be\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7"
Sep 15 21:10:22.582277 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.582246    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c11383a-a64c-49e0-9b91-110607ed70be-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7\" (UID: \"3c11383a-a64c-49e0-9b91-110607ed70be\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7"
Sep 15 21:10:22.582960 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.582932    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c11383a-a64c-49e0-9b91-110607ed70be-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7\" (UID: \"3c11383a-a64c-49e0-9b91-110607ed70be\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7"
Sep 15 21:10:22.590933 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.590907    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tggmb\" (UniqueName: \"kubernetes.io/projected/3c11383a-a64c-49e0-9b91-110607ed70be-kube-api-access-tggmb\") pod \"payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7\" (UID: \"3c11383a-a64c-49e0-9b91-110607ed70be\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7"
Sep 15 21:10:22.631558 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.631520    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw" event={"ID":"98b8aab3-7b1d-4723-b152-ec7f53357a2b","Type":"ContainerStarted","Data":"3e48606358f303e54ca301d615ff5b6588412ba4f1bfec9ca71ec720bf8994f1"}
Sep 15 21:10:22.662812 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.662771    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7"
Sep 15 21:10:22.851447 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.851410    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"]
Sep 15 21:10:22.864010 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.862398    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"
Sep 15 21:10:22.867465 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.867091    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"]
Sep 15 21:10:22.867465 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.867361    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-0df719-serving-cert\""
Sep 15 21:10:22.887657 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.887622    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:22.938877 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.938851    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7"]
Sep 15 21:10:22.941335 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:10:22.941308    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c11383a_a64c_49e0_9b91_110607ed70be.slice/crio-cbb70cabcd7863b3f052c52ee774c72d61e82eee8279399638e3f5c0e81826f3 WatchSource:0}: Error finding container cbb70cabcd7863b3f052c52ee774c72d61e82eee8279399638e3f5c0e81826f3: Status 404 returned error can't find the container with id cbb70cabcd7863b3f052c52ee774c72d61e82eee8279399638e3f5c0e81826f3
Sep 15 21:10:22.967404 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.967361    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:22.986686 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.986655    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fv4k\" (UniqueName: \"kubernetes.io/projected/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-kube-api-access-8fv4k\") pod \"maas-api-e2e-inf-a-0df719-796f76c855-jg8xk\" (UID: \"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"
Sep 15 21:10:22.986868 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.986703    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0df719-796f76c855-jg8xk\" (UID: \"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"
Sep 15 21:10:22.986868 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.986828    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-metrics-certs\") pod \"maas-api-e2e-inf-a-0df719-796f76c855-jg8xk\" (UID: \"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"
Sep 15 21:10:22.986868 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:22.986863    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-maas-api-tls\") pod \"maas-api-e2e-inf-a-0df719-796f76c855-jg8xk\" (UID: \"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"
Sep 15 21:10:23.088178 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:23.088138    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fv4k\" (UniqueName: \"kubernetes.io/projected/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-kube-api-access-8fv4k\") pod \"maas-api-e2e-inf-a-0df719-796f76c855-jg8xk\" (UID: \"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"
Sep 15 21:10:23.088371 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:23.088191    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0df719-796f76c855-jg8xk\" (UID: \"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"
Sep 15 21:10:23.088371 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:23.088292    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-metrics-certs\") pod \"maas-api-e2e-inf-a-0df719-796f76c855-jg8xk\" (UID: \"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"
Sep 15 21:10:23.088371 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:23.088322    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-maas-api-tls\") pod \"maas-api-e2e-inf-a-0df719-796f76c855-jg8xk\" (UID: \"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"
Sep 15 21:10:23.088629 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:10:23.088548    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-serving-cert: secret "maas-api-e2e-inf-a-0df719-serving-cert" not found
Sep 15 21:10:23.088695 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:10:23.088634    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-maas-api-tls podName:6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3 nodeName:}" failed. No retries permitted until 2026-09-15 21:10:23.58861057 +0000 UTC m=+760.130773199 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-maas-api-tls") pod "maas-api-e2e-inf-a-0df719-796f76c855-jg8xk" (UID: "6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3") : secret "maas-api-e2e-inf-a-0df719-serving-cert" not found
Sep 15 21:10:23.089205 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:23.089182    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0df719-796f76c855-jg8xk\" (UID: \"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"
Sep 15 21:10:23.091149 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:23.091108    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-metrics-certs\") pod \"maas-api-e2e-inf-a-0df719-796f76c855-jg8xk\" (UID: \"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"
Sep 15 21:10:23.098014 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:23.097977    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fv4k\" (UniqueName: \"kubernetes.io/projected/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-kube-api-access-8fv4k\") pod \"maas-api-e2e-inf-a-0df719-796f76c855-jg8xk\" (UID: \"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"
Sep 15 21:10:23.551254 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:23.551208    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:23.593975 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:23.593826    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-maas-api-tls\") pod \"maas-api-e2e-inf-a-0df719-796f76c855-jg8xk\" (UID: \"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"
Sep 15 21:10:23.597164 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:23.597088    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-maas-api-tls\") pod \"maas-api-e2e-inf-a-0df719-796f76c855-jg8xk\" (UID: \"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"
Sep 15 21:10:23.638802 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:23.638759    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw" event={"ID":"98b8aab3-7b1d-4723-b152-ec7f53357a2b","Type":"ContainerStarted","Data":"4f18fe9f386a40679ac13378ffbd140fa68c6f253bf9ee747da3ecaed480ce16"}
Sep 15 21:10:23.639010 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:23.638830    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"
Sep 15 21:10:23.640598 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:23.640572    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7" event={"ID":"3c11383a-a64c-49e0-9b91-110607ed70be","Type":"ContainerStarted","Data":"8beb229bfce94d1f3e58b57582fab9ae52d1c83779c0c025211075bc14893a3e"}
Sep 15 21:10:23.640742 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:23.640616    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7" event={"ID":"3c11383a-a64c-49e0-9b91-110607ed70be","Type":"ContainerStarted","Data":"cbb70cabcd7863b3f052c52ee774c72d61e82eee8279399638e3f5c0e81826f3"}
Sep 15 21:10:23.640742 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:23.640661    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7"
Sep 15 21:10:23.642139 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:23.642101    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:23.662389 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:23.662333    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw" podStartSLOduration=1.979950171 podStartE2EDuration="2.662316166s" totalImagesPullingTime="682.365995ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:10:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:10:23.658968537 +0000 UTC m=+760.201131204" watchObservedRunningTime="2026-09-15 21:10:23.662316166 +0000 UTC m=+760.204478838"
Sep 15 21:10:23.684965 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:23.684896    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7" podStartSLOduration=1.684871301 podStartE2EDuration="1.684871301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:10:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:10:23.682567168 +0000 UTC m=+760.224729829" watchObservedRunningTime="2026-09-15 21:10:23.684871301 +0000 UTC m=+760.227033940"
Sep 15 21:10:23.741144 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:23.741108    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:23.780240 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:23.780196    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"
Sep 15 21:10:24.068405 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:24.068382    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"]
Sep 15 21:10:24.070611 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:10:24.070580    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c99a9fa_0a11_4e5d_8e1d_1e02e46cf7f3.slice/crio-b823653b7c0389e56163d91ded011dd4d8350b27f5aa8658752374c71d45258b WatchSource:0}: Error finding container b823653b7c0389e56163d91ded011dd4d8350b27f5aa8658752374c71d45258b: Status 404 returned error can't find the container with id b823653b7c0389e56163d91ded011dd4d8350b27f5aa8658752374c71d45258b
Sep 15 21:10:24.416914 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:24.416875    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:24.646889 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:24.646850    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk" event={"ID":"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3","Type":"ContainerStarted","Data":"3cbb9e171211939f2d93f110eef81dff4bcd86cf60a80da1cc6f160f917ab091"}
Sep 15 21:10:24.647304 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:24.646917    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk" event={"ID":"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3","Type":"ContainerStarted","Data":"b823653b7c0389e56163d91ded011dd4d8350b27f5aa8658752374c71d45258b"}
Sep 15 21:10:24.647304 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:24.646970    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"
Sep 15 21:10:24.667990 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:24.667898    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk" podStartSLOduration=2.210968142 podStartE2EDuration="2.667881107s" totalImagesPullingTime="456.912965ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:10:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:10:24.666025237 +0000 UTC m=+761.208187882" watchObservedRunningTime="2026-09-15 21:10:24.667881107 +0000 UTC m=+761.210043739"
Sep 15 21:10:25.305483 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:25.305452    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:26.161611 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:26.161578    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:26.225715 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:26.225679    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:26.302875 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:26.302844    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:26.405328 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:26.405297    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:26.722290 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:26.722232    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:27.147912 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.147867    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"]
Sep 15 21:10:27.152323 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.152296    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"
Sep 15 21:10:27.155566 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.155546    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-544890-serving-cert\""
Sep 15 21:10:27.167595 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.167232    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"]
Sep 15 21:10:27.191239 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.191180    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:27.331953 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.331907    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-maas-api-tls\") pod \"maas-api-e2e-worker-w0-544890-7756874b8c-2vqds\" (UID: \"b27dd28b-9a41-4a3e-a821-12ad8fb7a413\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"
Sep 15 21:10:27.332167 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.331960    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-544890-7756874b8c-2vqds\" (UID: \"b27dd28b-9a41-4a3e-a821-12ad8fb7a413\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"
Sep 15 21:10:27.332167 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.332025    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-metrics-certs\") pod \"maas-api-e2e-worker-w0-544890-7756874b8c-2vqds\" (UID: \"b27dd28b-9a41-4a3e-a821-12ad8fb7a413\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"
Sep 15 21:10:27.332167 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.332116    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4799p\" (UniqueName: \"kubernetes.io/projected/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-kube-api-access-4799p\") pod \"maas-api-e2e-worker-w0-544890-7756874b8c-2vqds\" (UID: \"b27dd28b-9a41-4a3e-a821-12ad8fb7a413\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"
Sep 15 21:10:27.396615 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.396582    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:27.432756 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.432650    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4799p\" (UniqueName: \"kubernetes.io/projected/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-kube-api-access-4799p\") pod \"maas-api-e2e-worker-w0-544890-7756874b8c-2vqds\" (UID: \"b27dd28b-9a41-4a3e-a821-12ad8fb7a413\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"
Sep 15 21:10:27.432756 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.432701    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-maas-api-tls\") pod \"maas-api-e2e-worker-w0-544890-7756874b8c-2vqds\" (UID: \"b27dd28b-9a41-4a3e-a821-12ad8fb7a413\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"
Sep 15 21:10:27.432756 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.432726    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-544890-7756874b8c-2vqds\" (UID: \"b27dd28b-9a41-4a3e-a821-12ad8fb7a413\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"
Sep 15 21:10:27.432756 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.432757    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-metrics-certs\") pod \"maas-api-e2e-worker-w0-544890-7756874b8c-2vqds\" (UID: \"b27dd28b-9a41-4a3e-a821-12ad8fb7a413\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"
Sep 15 21:10:27.433117 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:10:27.432829    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-serving-cert: secret "maas-api-e2e-worker-w0-544890-serving-cert" not found
Sep 15 21:10:27.433117 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:10:27.432911    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-maas-api-tls podName:b27dd28b-9a41-4a3e-a821-12ad8fb7a413 nodeName:}" failed. No retries permitted until 2026-09-15 21:10:27.932890152 +0000 UTC m=+764.475052765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-maas-api-tls") pod "maas-api-e2e-worker-w0-544890-7756874b8c-2vqds" (UID: "b27dd28b-9a41-4a3e-a821-12ad8fb7a413") : secret "maas-api-e2e-worker-w0-544890-serving-cert" not found
Sep 15 21:10:27.433617 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.433478    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-544890-7756874b8c-2vqds\" (UID: \"b27dd28b-9a41-4a3e-a821-12ad8fb7a413\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"
Sep 15 21:10:27.435387 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.435367    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-metrics-certs\") pod \"maas-api-e2e-worker-w0-544890-7756874b8c-2vqds\" (UID: \"b27dd28b-9a41-4a3e-a821-12ad8fb7a413\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"
Sep 15 21:10:27.441922 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.441895    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4799p\" (UniqueName: \"kubernetes.io/projected/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-kube-api-access-4799p\") pod \"maas-api-e2e-worker-w0-544890-7756874b8c-2vqds\" (UID: \"b27dd28b-9a41-4a3e-a821-12ad8fb7a413\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"
Sep 15 21:10:27.720563 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.720452    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:27.819883 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.819837    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:27.837567 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.837533    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2"]
Sep 15 21:10:27.841282 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.841266    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2"
Sep 15 21:10:27.844223 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.844199    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-544890\""
Sep 15 21:10:27.855333 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.855307    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2"]
Sep 15 21:10:27.860323 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.860298    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v"]
Sep 15 21:10:27.863862 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.863844    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v"
Sep 15 21:10:27.867162 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.867144    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-544890-dockercfg-rxwnq\""
Sep 15 21:10:27.880051 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.880024    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v"]
Sep 15 21:10:27.937156 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.937124    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-maas-api-tls\") pod \"maas-api-e2e-worker-w0-544890-7756874b8c-2vqds\" (UID: \"b27dd28b-9a41-4a3e-a821-12ad8fb7a413\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"
Sep 15 21:10:27.937344 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.937174    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkqgk\" (UniqueName: \"kubernetes.io/projected/c8779115-0e1b-4b97-81c6-501c16ce4412-kube-api-access-qkqgk\") pod \"payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2\" (UID: \"c8779115-0e1b-4b97-81c6-501c16ce4412\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2"
Sep 15 21:10:27.937344 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.937219    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8779115-0e1b-4b97-81c6-501c16ce4412-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2\" (UID: \"c8779115-0e1b-4b97-81c6-501c16ce4412\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2"
Sep 15 21:10:27.939778 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:27.939753    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-maas-api-tls\") pod \"maas-api-e2e-worker-w0-544890-7756874b8c-2vqds\" (UID: \"b27dd28b-9a41-4a3e-a821-12ad8fb7a413\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"
Sep 15 21:10:28.037935 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.037846    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5a525f4-a59e-4444-a998-453225e4f1e4-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v\" (UID: \"f5a525f4-a59e-4444-a998-453225e4f1e4\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v"
Sep 15 21:10:28.037935 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.037915    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkqgk\" (UniqueName: \"kubernetes.io/projected/c8779115-0e1b-4b97-81c6-501c16ce4412-kube-api-access-qkqgk\") pod \"payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2\" (UID: \"c8779115-0e1b-4b97-81c6-501c16ce4412\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2"
Sep 15 21:10:28.038147 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.037963    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4shj\" (UniqueName: \"kubernetes.io/projected/f5a525f4-a59e-4444-a998-453225e4f1e4-kube-api-access-v4shj\") pod \"payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v\" (UID: \"f5a525f4-a59e-4444-a998-453225e4f1e4\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v"
Sep 15 21:10:28.038147 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.037984    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8779115-0e1b-4b97-81c6-501c16ce4412-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2\" (UID: \"c8779115-0e1b-4b97-81c6-501c16ce4412\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2"
Sep 15 21:10:28.038726 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.038705    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8779115-0e1b-4b97-81c6-501c16ce4412-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2\" (UID: \"c8779115-0e1b-4b97-81c6-501c16ce4412\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2"
Sep 15 21:10:28.046505 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.046473    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkqgk\" (UniqueName: \"kubernetes.io/projected/c8779115-0e1b-4b97-81c6-501c16ce4412-kube-api-access-qkqgk\") pod \"payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2\" (UID: \"c8779115-0e1b-4b97-81c6-501c16ce4412\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2"
Sep 15 21:10:28.071393 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.071355    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"
Sep 15 21:10:28.139157 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.139124    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4shj\" (UniqueName: \"kubernetes.io/projected/f5a525f4-a59e-4444-a998-453225e4f1e4-kube-api-access-v4shj\") pod \"payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v\" (UID: \"f5a525f4-a59e-4444-a998-453225e4f1e4\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v"
Sep 15 21:10:28.139323 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.139244    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5a525f4-a59e-4444-a998-453225e4f1e4-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v\" (UID: \"f5a525f4-a59e-4444-a998-453225e4f1e4\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v"
Sep 15 21:10:28.140530 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.140449    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5a525f4-a59e-4444-a998-453225e4f1e4-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v\" (UID: \"f5a525f4-a59e-4444-a998-453225e4f1e4\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v"
Sep 15 21:10:28.148259 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.148235    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4shj\" (UniqueName: \"kubernetes.io/projected/f5a525f4-a59e-4444-a998-453225e4f1e4-kube-api-access-v4shj\") pod \"payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v\" (UID: \"f5a525f4-a59e-4444-a998-453225e4f1e4\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v"
Sep 15 21:10:28.155119 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.155081    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2"
Sep 15 21:10:28.176434 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.176399    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v"
Sep 15 21:10:28.350671 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.350648    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"]
Sep 15 21:10:28.351462 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:10:28.351435    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb27dd28b_9a41_4a3e_a821_12ad8fb7a413.slice/crio-33b07a932ebfd18cbc1f838933af4190ffcaba7efd07adba20dddd2a3d786bfc WatchSource:0}: Error finding container 33b07a932ebfd18cbc1f838933af4190ffcaba7efd07adba20dddd2a3d786bfc: Status 404 returned error can't find the container with id 33b07a932ebfd18cbc1f838933af4190ffcaba7efd07adba20dddd2a3d786bfc
Sep 15 21:10:28.438514 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.438464    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2"]
Sep 15 21:10:28.440250 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:10:28.440215    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8779115_0e1b_4b97_81c6_501c16ce4412.slice/crio-4809072c086a185d12c8cb6209657ef7a5896987ec84cd800d16d7215e760fa9 WatchSource:0}: Error finding container 4809072c086a185d12c8cb6209657ef7a5896987ec84cd800d16d7215e760fa9: Status 404 returned error can't find the container with id 4809072c086a185d12c8cb6209657ef7a5896987ec84cd800d16d7215e760fa9
Sep 15 21:10:28.473856 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.473823    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v"]
Sep 15 21:10:28.479763 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:10:28.479737    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5a525f4_a59e_4444_a998_453225e4f1e4.slice/crio-5a8f3cdef2a34833e4e5043de7f1d8c31c93f41be8e410d68c2328434bd8f2be WatchSource:0}: Error finding container 5a8f3cdef2a34833e4e5043de7f1d8c31c93f41be8e410d68c2328434bd8f2be: Status 404 returned error can't find the container with id 5a8f3cdef2a34833e4e5043de7f1d8c31c93f41be8e410d68c2328434bd8f2be
Sep 15 21:10:28.669014 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.668905    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v" event={"ID":"f5a525f4-a59e-4444-a998-453225e4f1e4","Type":"ContainerStarted","Data":"695b96c29d40700fc14143ed0ba9772dfd019c4f876c8611f0d33c1f0bcb4a5d"}
Sep 15 21:10:28.669014 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.668969    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v" event={"ID":"f5a525f4-a59e-4444-a998-453225e4f1e4","Type":"ContainerStarted","Data":"5a8f3cdef2a34833e4e5043de7f1d8c31c93f41be8e410d68c2328434bd8f2be"}
Sep 15 21:10:28.669250 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.669087    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v"
Sep 15 21:10:28.670727 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.670705    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2" event={"ID":"c8779115-0e1b-4b97-81c6-501c16ce4412","Type":"ContainerStarted","Data":"2bb99b311c4c698ef5e8a5c3ead60e465d0e877f9448ffa3c9143580c50f005d"}
Sep 15 21:10:28.670841 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.670746    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2" event={"ID":"c8779115-0e1b-4b97-81c6-501c16ce4412","Type":"ContainerStarted","Data":"4809072c086a185d12c8cb6209657ef7a5896987ec84cd800d16d7215e760fa9"}
Sep 15 21:10:28.670841 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.670779    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2"
Sep 15 21:10:28.671976 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.671954    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds" event={"ID":"b27dd28b-9a41-4a3e-a821-12ad8fb7a413","Type":"ContainerStarted","Data":"33b07a932ebfd18cbc1f838933af4190ffcaba7efd07adba20dddd2a3d786bfc"}
Sep 15 21:10:28.687852 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.687796    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v" podStartSLOduration=1.6877805019999998 podStartE2EDuration="1.687780502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:10:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:10:28.686899973 +0000 UTC m=+765.229062635" watchObservedRunningTime="2026-09-15 21:10:28.687780502 +0000 UTC m=+765.229943137"
Sep 15 21:10:28.710764 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:28.710705    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2" podStartSLOduration=1.710684318 podStartE2EDuration="1.710684318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:10:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:10:28.707344749 +0000 UTC m=+765.249507432" watchObservedRunningTime="2026-09-15 21:10:28.710684318 +0000 UTC m=+765.252846957"
Sep 15 21:10:29.653451 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:29.653419    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"
Sep 15 21:10:29.679256 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:29.679201    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds" event={"ID":"b27dd28b-9a41-4a3e-a821-12ad8fb7a413","Type":"ContainerStarted","Data":"ed5aef2f3cdb674d2237970a80c3c93470cca7d76c5669e82f45d11440c36dee"}
Sep 15 21:10:29.706357 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:29.705782    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:29.709650 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:29.709602    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds" podStartSLOduration=1.999795888 podStartE2EDuration="2.70958474s" totalImagesPullingTime="709.788852ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:10:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:10:29.70735746 +0000 UTC m=+766.249520087" watchObservedRunningTime="2026-09-15 21:10:29.70958474 +0000 UTC m=+766.251747375"
Sep 15 21:10:29.796123 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:29.796088    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:30.658022 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:30.657994    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"
Sep 15 21:10:30.685139 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:30.685111    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"
Sep 15 21:10:30.751912 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:30.751877    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:31.767989 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:31.767938    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:32.651363 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:32.651330    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:33.400249 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:33.400212    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:33.500675 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:33.500627    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:33.933907 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:33.933870    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:34.028830 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:34.028782    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:34.606001 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:34.605968    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:34.649364 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:34.649333    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7"
Sep 15 21:10:34.844366 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:34.844327    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:34.945220 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:34.945131    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:35.352378 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:35.352347    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:35.477672 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:35.477632    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:35.677974 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:35.677877    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:35.860303 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:35.860269    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:35.965849 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:35.965755    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:36.107919 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:36.107882    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:36.216054 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:36.215956    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:36.326659 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:36.326615    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:36.509755 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:36.509661    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:36.697167 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:36.697134    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"
Sep 15 21:10:36.911313 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:36.911282    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:37.287369 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:37.287272    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:37.624614 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:37.624570    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:37.720672 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:37.720634    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:38.442820 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:38.442769    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:38.885819 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:38.885773    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:39.456975 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:39.456937    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:39.681411 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:39.681379    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2"
Sep 15 21:10:39.681628 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:39.681538    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v"
Sep 15 21:10:39.796448 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:39.796372    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:39.910136 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:39.910096    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:40.474567 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:40.474527    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:40.720461 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:40.720428    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:40.830595 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:40.830479    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:40.937553 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:40.937484    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:41.588356 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:41.588316    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:42.583437 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:42.583396    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:43.348614 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:43.348578    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"]
Sep 15 21:10:43.353436 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:43.353409    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"
Sep 15 21:10:43.356164 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:43.356140    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-253ca1-serving-cert\""
Sep 15 21:10:43.364534 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:43.364149    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d03e0f2-93b3-497a-99db-d4c33188e293-openshift-service-ca\") pod \"maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7\" (UID: \"0d03e0f2-93b3-497a-99db-d4c33188e293\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"
Sep 15 21:10:43.364534 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:43.364200    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d03e0f2-93b3-497a-99db-d4c33188e293-maas-api-tls\") pod \"maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7\" (UID: \"0d03e0f2-93b3-497a-99db-d4c33188e293\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"
Sep 15 21:10:43.364534 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:43.364242    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d03e0f2-93b3-497a-99db-d4c33188e293-metrics-certs\") pod \"maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7\" (UID: \"0d03e0f2-93b3-497a-99db-d4c33188e293\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"
Sep 15 21:10:43.364534 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:43.364291    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh5h6\" (UniqueName: \"kubernetes.io/projected/0d03e0f2-93b3-497a-99db-d4c33188e293-kube-api-access-qh5h6\") pod \"maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7\" (UID: \"0d03e0f2-93b3-497a-99db-d4c33188e293\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"
Sep 15 21:10:43.365811 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:43.365336    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"]
Sep 15 21:10:43.392724 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:43.392687    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:43.459659 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:43.459629    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:43.465805 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:43.465768    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d03e0f2-93b3-497a-99db-d4c33188e293-openshift-service-ca\") pod \"maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7\" (UID: \"0d03e0f2-93b3-497a-99db-d4c33188e293\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"
Sep 15 21:10:43.465993 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:43.465811    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d03e0f2-93b3-497a-99db-d4c33188e293-maas-api-tls\") pod \"maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7\" (UID: \"0d03e0f2-93b3-497a-99db-d4c33188e293\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"
Sep 15 21:10:43.465993 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:43.465838    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d03e0f2-93b3-497a-99db-d4c33188e293-metrics-certs\") pod \"maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7\" (UID: \"0d03e0f2-93b3-497a-99db-d4c33188e293\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"
Sep 15 21:10:43.465993 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:43.465869    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh5h6\" (UniqueName: \"kubernetes.io/projected/0d03e0f2-93b3-497a-99db-d4c33188e293-kube-api-access-qh5h6\") pod \"maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7\" (UID: \"0d03e0f2-93b3-497a-99db-d4c33188e293\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"
Sep 15 21:10:43.466509 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:43.466469    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d03e0f2-93b3-497a-99db-d4c33188e293-openshift-service-ca\") pod \"maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7\" (UID: \"0d03e0f2-93b3-497a-99db-d4c33188e293\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"
Sep 15 21:10:43.468620 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:43.468600    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d03e0f2-93b3-497a-99db-d4c33188e293-metrics-certs\") pod \"maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7\" (UID: \"0d03e0f2-93b3-497a-99db-d4c33188e293\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"
Sep 15 21:10:43.468681 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:43.468636    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d03e0f2-93b3-497a-99db-d4c33188e293-maas-api-tls\") pod \"maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7\" (UID: \"0d03e0f2-93b3-497a-99db-d4c33188e293\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"
Sep 15 21:10:43.478137 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:43.478107    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh5h6\" (UniqueName: \"kubernetes.io/projected/0d03e0f2-93b3-497a-99db-d4c33188e293-kube-api-access-qh5h6\") pod \"maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7\" (UID: \"0d03e0f2-93b3-497a-99db-d4c33188e293\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"
Sep 15 21:10:43.559759 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:43.559709    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:43.671148 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:43.671046    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"
Sep 15 21:10:43.959373 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:43.959335    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"]
Sep 15 21:10:43.961149 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:10:43.961118    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d03e0f2_93b3_497a_99db_d4c33188e293.slice/crio-98a3f09843688e69ddcafd8a33a1b0e2a9d931bddfc7027acbc577d4301ec61d WatchSource:0}: Error finding container 98a3f09843688e69ddcafd8a33a1b0e2a9d931bddfc7027acbc577d4301ec61d: Status 404 returned error can't find the container with id 98a3f09843688e69ddcafd8a33a1b0e2a9d931bddfc7027acbc577d4301ec61d
Sep 15 21:10:43.962149 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:43.962118    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:44.070340 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:44.070292    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:44.167859 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:44.167811    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:44.248224 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:44.248136    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:44.252986 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:44.252946    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv"]
Sep 15 21:10:44.257898 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:44.257873    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv"
Sep 15 21:10:44.260638 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:44.260614    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-253ca1\""
Sep 15 21:10:44.260788 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:44.260687    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-253ca1-dockercfg-vbgvr\""
Sep 15 21:10:44.267433 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:44.267406    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv"]
Sep 15 21:10:44.273913 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:44.273889    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k47gn\" (UniqueName: \"kubernetes.io/projected/b67c890d-216f-4e2f-bdd7-5b587e49ccb1-kube-api-access-k47gn\") pod \"payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv\" (UID: \"b67c890d-216f-4e2f-bdd7-5b587e49ccb1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv"
Sep 15 21:10:44.274032 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:44.273940    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b67c890d-216f-4e2f-bdd7-5b587e49ccb1-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv\" (UID: \"b67c890d-216f-4e2f-bdd7-5b587e49ccb1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv"
Sep 15 21:10:44.375133 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:44.375104    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k47gn\" (UniqueName: \"kubernetes.io/projected/b67c890d-216f-4e2f-bdd7-5b587e49ccb1-kube-api-access-k47gn\") pod \"payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv\" (UID: \"b67c890d-216f-4e2f-bdd7-5b587e49ccb1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv"
Sep 15 21:10:44.375261 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:44.375173    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b67c890d-216f-4e2f-bdd7-5b587e49ccb1-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv\" (UID: \"b67c890d-216f-4e2f-bdd7-5b587e49ccb1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv"
Sep 15 21:10:44.375952 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:44.375930    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b67c890d-216f-4e2f-bdd7-5b587e49ccb1-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv\" (UID: \"b67c890d-216f-4e2f-bdd7-5b587e49ccb1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv"
Sep 15 21:10:44.383292 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:44.383267    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k47gn\" (UniqueName: \"kubernetes.io/projected/b67c890d-216f-4e2f-bdd7-5b587e49ccb1-kube-api-access-k47gn\") pod \"payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv\" (UID: \"b67c890d-216f-4e2f-bdd7-5b587e49ccb1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv"
Sep 15 21:10:44.570091 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:44.569994    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv"
Sep 15 21:10:44.763930 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:44.763893    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:44.769118 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:44.769089    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7" event={"ID":"0d03e0f2-93b3-497a-99db-d4c33188e293","Type":"ContainerStarted","Data":"b7f3681b3481251682fbb54714edbeaf732395dd8fa2ab19b4b90c2ff4c3d524"}
Sep 15 21:10:44.769308 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:44.769140    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7" event={"ID":"0d03e0f2-93b3-497a-99db-d4c33188e293","Type":"ContainerStarted","Data":"98a3f09843688e69ddcafd8a33a1b0e2a9d931bddfc7027acbc577d4301ec61d"}
Sep 15 21:10:44.769308 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:44.769162    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"
Sep 15 21:10:44.789478 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:44.789423    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7" podStartSLOduration=1.43958817 podStartE2EDuration="1.789407828s" totalImagesPullingTime="349.819658ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:10:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:10:44.787053826 +0000 UTC m=+781.329216507" watchObservedRunningTime="2026-09-15 21:10:44.789407828 +0000 UTC m=+781.331570462"
Sep 15 21:10:44.854893 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:44.854863    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv"]
Sep 15 21:10:44.855265 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:10:44.855242    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb67c890d_216f_4e2f_bdd7_5b587e49ccb1.slice/crio-5d150a837be15b8791b6164b7951507924d6b72fd2a318c58b4488704c318869 WatchSource:0}: Error finding container 5d150a837be15b8791b6164b7951507924d6b72fd2a318c58b4488704c318869: Status 404 returned error can't find the container with id 5d150a837be15b8791b6164b7951507924d6b72fd2a318c58b4488704c318869
Sep 15 21:10:45.485802 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:45.485747    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:45.585830 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:45.585792    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:45.777786 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:45.777680    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv" event={"ID":"b67c890d-216f-4e2f-bdd7-5b587e49ccb1","Type":"ContainerStarted","Data":"c3d2f95326e772eaed8b4d7e74b3c3e59d0900189df747d749a91c3020256ccf"}
Sep 15 21:10:45.777786 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:45.777742    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv" event={"ID":"b67c890d-216f-4e2f-bdd7-5b587e49ccb1","Type":"ContainerStarted","Data":"5d150a837be15b8791b6164b7951507924d6b72fd2a318c58b4488704c318869"}
Sep 15 21:10:45.777786 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:45.777773    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv"
Sep 15 21:10:45.795279 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:45.795218    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv" podStartSLOduration=1.795200907 podStartE2EDuration="1.795200907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:10:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:10:45.795024708 +0000 UTC m=+782.337187360" watchObservedRunningTime="2026-09-15 21:10:45.795200907 +0000 UTC m=+782.337363597"
Sep 15 21:10:46.082974 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:46.082870    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:46.182756 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:46.182720    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:46.314322 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:46.314288    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:46.489664 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:46.489629    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:47.255376 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:47.255339    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:48.290443 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:48.290409    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:49.328522 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:49.328471    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:50.251820 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:50.251779    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:50.783197 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:50.783170    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"
Sep 15 21:10:51.005544 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:51.005477    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:52.008176 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:52.008148    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:52.775005 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:52.774976    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:53.792087 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:53.792048    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:54.499828 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:54.499790    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:54.596275 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:54.596238    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:55.489237 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:55.489204    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:56.284236 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:56.284198    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:56.785435 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:56.785407    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv"
Sep 15 21:10:57.386886 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:57.386849    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:58.286695 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:58.286657    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:58.460469 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:58.460416    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:58.570170 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:58.570085    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:59.388591 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:59.388553    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:10:59.490055 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:10:59.490017    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:00.195556 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:00.195517    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:00.486184 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:00.486074    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:01.079555 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:01.079526    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:01.774617 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:01.774578    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:01.878214 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:01.878181    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:01.978292 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:01.978254    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:02.120003 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:02.119963    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:02.998390 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:02.998347    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:03.744600 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:03.744554    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:03.853207 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:03.853154    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:03.953789 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:03.953747    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:04.052895 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:04.052810    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:04.186323 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:04.186285    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:04.255566 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:04.255521    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:04.640937 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:04.640901    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:04.788767 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:04.788673    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:04.884847 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:04.884816    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:05.100637 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:05.100598    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:05.207262 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:05.207224    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:05.363044 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:05.362956    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:06.165355 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:06.165313    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:06.984432 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:06.984397    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:07.484917 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:07.484886    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:07.589528 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:07.589458    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:07.990211 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:07.990179    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:08.958220 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:08.958189    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:09.085771 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:09.085723    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:09.205000 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:09.204955    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:09.337233 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:09.337143    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:10.097582 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:10.097541    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:10.883674 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:10.883636    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:10.957265 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:10.957208    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:11.370479 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:11.370442    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:11.528866 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:11.528830    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:11.629354 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:11.629266    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:12.067209 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:12.067103    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:12.889553 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:12.889522    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:13.628312 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:13.628281    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:15.050645 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:15.050611    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:15.121795 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:15.121765    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:33.358021 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:33.357978    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:34.182952 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:34.182913    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:34.878115 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:34.878080    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:35.689817 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:35.689772    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:36.371050 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:36.371007    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:37.076423 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.076377    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:37.490309 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.490274    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"]
Sep 15 21:11:37.496217 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.496194    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:37.500555 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.500101    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-544890\"/\"simulator-w0-544890-kserve-self-signed-certs\""
Sep 15 21:11:37.500555 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.500101    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-544890\"/\"openshift-service-ca.crt\""
Sep 15 21:11:37.500555 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.500200    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-544890\"/\"default-dockercfg-bxfmc\""
Sep 15 21:11:37.500555 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.500282    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-544890\"/\"kube-root-ca.crt\""
Sep 15 21:11:37.502948 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.502922    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"]
Sep 15 21:11:37.513839 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.513806    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:37.600744 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.600709    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-dshm\") pod \"simulator-w0-544890-kserve-5fd7dd49-lvgdn\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") " pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:37.600925 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.600771    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-tmp-dir\") pod \"simulator-w0-544890-kserve-5fd7dd49-lvgdn\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") " pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:37.600925 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.600869    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6702c4b7-5609-496b-808f-9a227938f6ad-tls-certs\") pod \"simulator-w0-544890-kserve-5fd7dd49-lvgdn\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") " pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:37.600925 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.600914    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29q5t\" (UniqueName: \"kubernetes.io/projected/6702c4b7-5609-496b-808f-9a227938f6ad-kube-api-access-29q5t\") pod \"simulator-w0-544890-kserve-5fd7dd49-lvgdn\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") " pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:37.601104 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.600946    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-home\") pod \"simulator-w0-544890-kserve-5fd7dd49-lvgdn\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") " pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:37.601160 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.601110    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-model-cache\") pod \"simulator-w0-544890-kserve-5fd7dd49-lvgdn\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") " pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:37.701881 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.701837    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-tmp-dir\") pod \"simulator-w0-544890-kserve-5fd7dd49-lvgdn\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") " pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:37.702063 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.701886    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6702c4b7-5609-496b-808f-9a227938f6ad-tls-certs\") pod \"simulator-w0-544890-kserve-5fd7dd49-lvgdn\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") " pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:37.702063 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.701922    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29q5t\" (UniqueName: \"kubernetes.io/projected/6702c4b7-5609-496b-808f-9a227938f6ad-kube-api-access-29q5t\") pod \"simulator-w0-544890-kserve-5fd7dd49-lvgdn\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") " pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:37.702063 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.701949    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-home\") pod \"simulator-w0-544890-kserve-5fd7dd49-lvgdn\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") " pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:37.702063 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.702057    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-model-cache\") pod \"simulator-w0-544890-kserve-5fd7dd49-lvgdn\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") " pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:37.702262 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.702086    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-dshm\") pod \"simulator-w0-544890-kserve-5fd7dd49-lvgdn\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") " pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:37.702323 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.702304    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-tmp-dir\") pod \"simulator-w0-544890-kserve-5fd7dd49-lvgdn\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") " pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:37.702382 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.702353    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-home\") pod \"simulator-w0-544890-kserve-5fd7dd49-lvgdn\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") " pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:37.702464 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.702441    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-model-cache\") pod \"simulator-w0-544890-kserve-5fd7dd49-lvgdn\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") " pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:37.704557 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.704505    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-dshm\") pod \"simulator-w0-544890-kserve-5fd7dd49-lvgdn\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") " pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:37.704708 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.704691    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6702c4b7-5609-496b-808f-9a227938f6ad-tls-certs\") pod \"simulator-w0-544890-kserve-5fd7dd49-lvgdn\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") " pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:37.710323 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.710296    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29q5t\" (UniqueName: \"kubernetes.io/projected/6702c4b7-5609-496b-808f-9a227938f6ad-kube-api-access-29q5t\") pod \"simulator-w0-544890-kserve-5fd7dd49-lvgdn\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") " pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:37.807248 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:37.807158    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:38.085772 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:38.085743    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"]
Sep 15 21:11:38.087834 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:11:38.087786    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6702c4b7_5609_496b_808f_9a227938f6ad.slice/crio-2afa5913e48c771113341b6d99122c5db2cd869b2eaf3bb4bf2ad196aced6aad WatchSource:0}: Error finding container 2afa5913e48c771113341b6d99122c5db2cd869b2eaf3bb4bf2ad196aced6aad: Status 404 returned error can't find the container with id 2afa5913e48c771113341b6d99122c5db2cd869b2eaf3bb4bf2ad196aced6aad
Sep 15 21:11:38.148277 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:38.148233    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:38.862348 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:38.862291    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:39.072871 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:39.072828    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn" event={"ID":"6702c4b7-5609-496b-808f-9a227938f6ad","Type":"ContainerStarted","Data":"b8d656a0639ea4b149472189a6d540330bc8b657c0de90590007285e62c8eafe"}
Sep 15 21:11:39.073065 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:39.072883    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn" event={"ID":"6702c4b7-5609-496b-808f-9a227938f6ad","Type":"ContainerStarted","Data":"2afa5913e48c771113341b6d99122c5db2cd869b2eaf3bb4bf2ad196aced6aad"}
Sep 15 21:11:39.089919 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:39.089862    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn" podStartSLOduration=2.089846768 podStartE2EDuration="2.089846768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:11:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:11:39.089676194 +0000 UTC m=+835.631838874" watchObservedRunningTime="2026-09-15 21:11:39.089846768 +0000 UTC m=+835.632009402"
Sep 15 21:11:39.576328 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:39.576290    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:40.375756 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:40.375720    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:41.074531 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:41.074471    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:41.881788 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:41.881747    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:42.571557 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:42.571526    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:47.807541 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:47.807503    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:47.807965 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:47.807555    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:47.820506 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:47.820456    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:48.129082 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:48.129052    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:11:48.153058 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:48.153024    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:48.766626 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:48.766588    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:48.836282 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:48.836241    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:48.936564 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:48.936524    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:48.995241 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:48.995202    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:49.112867 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:49.112833    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:49.340642 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:49.340462    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:49.548956 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:49.548856    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:49.889873 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:49.889811    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:50.051824 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:50.051788    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:50.239690 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:50.239602    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:51.000870 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:51.000837    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:51.846392 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:51.846353    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:52.549628 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:52.549569    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:52.649809 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:52.649770    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:52.790547 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:52.790467    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:53.703804 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:53.703763    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:54.708229 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:54.708191    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:54.809609 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:54.809565    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:54.948251 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:54.948215    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:55.340214 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:55.340178    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:55.439505 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:55.439448    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:55.790574 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:55.790544    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:56.525392 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:56.525360    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:57.248909 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:57.248868    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:11:58.077707 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:11:58.077675    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:09.373953 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:09.373911    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:14.576404 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:14.576371    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:15.874977 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:15.874934    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:17.149717 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:17.149679    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:17.966056 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:17.966023    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:18.986916 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:18.986879    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:19.878066 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:19.878024    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:20.672406 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:20.672372    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:21.476639 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:21.476603    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:22.154766 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:22.154704    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:22.827440 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:22.827403    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:23.744310 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:23.744274    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:23.898617 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:23.898581    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:24.377899 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:24.377867    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:24.475076 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:24.475036    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:24.573644 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:24.573606    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:24.651964 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:24.651875    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:24.983903 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:24.983813    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:25.745386 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:25.745343    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:25.845421 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:25.845385    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:26.223208 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:26.223155    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:26.480338 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:26.480231    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:26.662760 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:26.662722    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:26.819855 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:26.819767    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:27.685936 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:27.685895    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:28.519075 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:28.519041    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:28.891971 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:28.891916    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:28.988246 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:28.988208    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:29.499322 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:29.499278    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:30.346011 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:30.345966    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:30.447828 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:30.447789    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:30.788129 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:30.788089    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:31.557918 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:31.557877    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:31.657557 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:31.657472    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:32.235234 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:32.235190    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:32.967826 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:32.967793    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:35.003382 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:35.003285    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:43.943158 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:43.943130    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-7t46m_c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf/console-operator/1.log"
Sep 15 21:12:43.945888 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:43.945865    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-7t46m_c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf/console-operator/1.log"
Sep 15 21:12:43.946290 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:43.946274    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4c7k_fc205950-caf0-4c47-84ce-a8fc54a1afb2/ovn-acl-logging/0.log"
Sep 15 21:12:43.948819 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:43.948800    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4c7k_fc205950-caf0-4c47-84ce-a8fc54a1afb2/ovn-acl-logging/0.log"
Sep 15 21:12:50.045615 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:50.045580    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:50.146690 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:50.146597    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:52.671880 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:52.671839    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:53.478388 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:53.478320    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:54.296792 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:54.296760    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:55.133034 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:55.132993    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:55.952786 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:55.952754    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:56.569565 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:56.569527    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:57.384297 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:57.384262    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:58.089611 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:58.089576    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:12:58.862215 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:12:58.862181    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:00.178193 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:00.178155    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:00.776726 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:00.776694    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:00.864233 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:00.864201    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:00.930869 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:00.930831    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:00.998465 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:00.998422    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:01.074557 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:01.074459    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:01.155668 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:01.155626    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:01.489823 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:01.489795    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:01.908430 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:01.908382    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:02.017196 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:02.017151    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:02.663120 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:02.663079    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:03.429449 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:03.429404    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:03.584830 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:03.584790    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:04.481127 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:04.481093    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:05.365397 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:05.365352    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:06.179935 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:06.179896    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:09.387165 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:09.387127    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:09.501935 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:09.501898    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:09.587963 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:09.587926    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:09.692001 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:09.691916    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:10.123477 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:10.123440    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:27.495738 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:27.495695    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:35.979503 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:35.979456    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:41.574016 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:41.573961    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:41.711010 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:41.710975    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:41.806465 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:41.806426    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:43.277809 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:43.277775    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:45.309046 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:45.309007    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:48.117855 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:48.117817    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:13:57.559058 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:13:57.559024    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:01.177126 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:01.177078    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:01.999911 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:01.999882    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:02.816967 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:02.816919    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:03.665695 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:03.665649    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:03.770924 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:03.770889    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:03.909215 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:03.909184    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:04.694802 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:04.694713    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:05.466033 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:05.465990    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:06.287806 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:06.287765    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:11.407955 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:11.407909    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:11.491629 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:11.491589    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:11.593215 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:11.593167    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:11.687538 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:11.687416    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:12.004315 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:12.004214    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:12.104869 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:12.104837    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:12.947738 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:12.947688    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:13.500661 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:13.500628    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:13.601565 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:13.601461    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:13.701414 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:13.701374    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:14.483625 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:14.483577    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:14.585731 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:14.585695    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:15.086317 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:15.086275    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:15.187854 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:15.187815    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:16.286281 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:16.286242    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:17.076717 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:17.076688    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:20.837606 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:20.837569    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:20.920004 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:20.919961    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:21.065290 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:21.065251    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:21.151359 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:21.151310    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:21.764998 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:21.764955    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:21.871770 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:21.871728    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:22.321680 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:22.321641    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:23.352646 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:23.352607    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:24.920745 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:24.920710    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:29.194447 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:29.194412    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:29.295527 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:29.295478    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:29.832028 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:29.831993    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:36.239558 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:36.239517    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:36.340264 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:36.340222    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:36.402365 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:36.402317    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:41.383032 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:41.382991    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:42.172185 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:42.172149    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:42.281541 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:42.281499    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:43.131533 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:43.131477    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:44.094157 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:44.094107    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:44.776685 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:44.776652    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:45.669367 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:45.669331    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:45.772791 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:45.772750    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:46.164147 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:46.164109    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:46.257838 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:46.257789    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:53.908231 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:53.908194    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:54.008037 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:54.007995    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:54.173845 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:54.173756    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:54.377737 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:54.377701    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:54.578778 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:54.578694    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:54.872895 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:54.872864    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:55.126141 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:55.126021    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:55.262049 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:55.262003    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:56.275533 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:56.275452    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:56.982628 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:56.982596    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:57.981765 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:57.981689    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:58.194411 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:58.194368    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:58.287219 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:58.287122    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:58.981221 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:58.981176    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:59.072736 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.072701    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:59.073539 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.073514    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"]
Sep 15 21:14:59.077200 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.077183    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:14:59.080196 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.080170    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-253ca1\"/\"test-model-253ca1-kserve-self-signed-certs\""
Sep 15 21:14:59.081162 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.081145    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-253ca1\"/\"default-dockercfg-5mfvz\""
Sep 15 21:14:59.081256 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.081170    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-253ca1\"/\"kube-root-ca.crt\""
Sep 15 21:14:59.081256 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.081218    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-253ca1\"/\"openshift-service-ca.crt\""
Sep 15 21:14:59.086057 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.086034    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"]
Sep 15 21:14:59.157079 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.157024    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-tmp-dir\") pod \"test-model-253ca1-kserve-684d8bfc9-pnmb7\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") " pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:14:59.157337 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.157216    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-model-cache\") pod \"test-model-253ca1-kserve-684d8bfc9-pnmb7\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") " pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:14:59.157337 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.157253    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1034c0f8-2cd5-42e5-889d-eb78749d96da-tls-certs\") pod \"test-model-253ca1-kserve-684d8bfc9-pnmb7\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") " pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:14:59.157337 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.157282    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zw65q\" (UniqueName: \"kubernetes.io/projected/1034c0f8-2cd5-42e5-889d-eb78749d96da-kube-api-access-zw65q\") pod \"test-model-253ca1-kserve-684d8bfc9-pnmb7\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") " pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:14:59.157337 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.157324    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-dshm\") pod \"test-model-253ca1-kserve-684d8bfc9-pnmb7\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") " pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:14:59.157595 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.157417    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-home\") pod \"test-model-253ca1-kserve-684d8bfc9-pnmb7\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") " pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:14:59.258555 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.258452    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-model-cache\") pod \"test-model-253ca1-kserve-684d8bfc9-pnmb7\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") " pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:14:59.258555 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.258516    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1034c0f8-2cd5-42e5-889d-eb78749d96da-tls-certs\") pod \"test-model-253ca1-kserve-684d8bfc9-pnmb7\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") " pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:14:59.258555 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.258552    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zw65q\" (UniqueName: \"kubernetes.io/projected/1034c0f8-2cd5-42e5-889d-eb78749d96da-kube-api-access-zw65q\") pod \"test-model-253ca1-kserve-684d8bfc9-pnmb7\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") " pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:14:59.258874 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.258589    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-dshm\") pod \"test-model-253ca1-kserve-684d8bfc9-pnmb7\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") " pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:14:59.258874 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.258635    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-home\") pod \"test-model-253ca1-kserve-684d8bfc9-pnmb7\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") " pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:14:59.258874 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.258675    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-tmp-dir\") pod \"test-model-253ca1-kserve-684d8bfc9-pnmb7\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") " pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:14:59.259036 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.258973    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-model-cache\") pod \"test-model-253ca1-kserve-684d8bfc9-pnmb7\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") " pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:14:59.259250 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.259228    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-home\") pod \"test-model-253ca1-kserve-684d8bfc9-pnmb7\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") " pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:14:59.259383 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.259282    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-tmp-dir\") pod \"test-model-253ca1-kserve-684d8bfc9-pnmb7\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") " pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:14:59.261374 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.261329    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-dshm\") pod \"test-model-253ca1-kserve-684d8bfc9-pnmb7\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") " pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:14:59.262108 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.262086    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1034c0f8-2cd5-42e5-889d-eb78749d96da-tls-certs\") pod \"test-model-253ca1-kserve-684d8bfc9-pnmb7\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") " pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:14:59.269517 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.269473    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zw65q\" (UniqueName: \"kubernetes.io/projected/1034c0f8-2cd5-42e5-889d-eb78749d96da-kube-api-access-zw65q\") pod \"test-model-253ca1-kserve-684d8bfc9-pnmb7\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") " pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:14:59.281268 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.281241    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:14:59.387932 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.387891    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:14:59.701574 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.701548    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"]
Sep 15 21:14:59.704431 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:14:59.704397    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1034c0f8_2cd5_42e5_889d_eb78749d96da.slice/crio-d8cbd937cab565f45a29da9262c8112deda6a19ff7a1f9a5de5269fd77904601 WatchSource:0}: Error finding container d8cbd937cab565f45a29da9262c8112deda6a19ff7a1f9a5de5269fd77904601: Status 404 returned error can't find the container with id d8cbd937cab565f45a29da9262c8112deda6a19ff7a1f9a5de5269fd77904601
Sep 15 21:14:59.851400 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:14:59.851367    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:00.132987 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:00.132949    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7" event={"ID":"1034c0f8-2cd5-42e5-889d-eb78749d96da","Type":"ContainerStarted","Data":"42174cc0830ba231a70691254bca159e313fc93d6aac78ec2cb8fb7a0965167b"}
Sep 15 21:15:00.133458 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:00.133004    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7" event={"ID":"1034c0f8-2cd5-42e5-889d-eb78749d96da","Type":"ContainerStarted","Data":"d8cbd937cab565f45a29da9262c8112deda6a19ff7a1f9a5de5269fd77904601"}
Sep 15 21:15:00.162826 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:00.162762    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7" podStartSLOduration=1.1627398 podStartE2EDuration="1.1627398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:14:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:15:00.161252581 +0000 UTC m=+1036.703415216" watchObservedRunningTime="2026-09-15 21:15:00.1627398 +0000 UTC m=+1036.704902435"
Sep 15 21:15:00.652106 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:00.652044    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:01.389356 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:01.389314    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:01.938671 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:01.938640    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:02.156103 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:02.156063    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:02.250414 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:02.250329    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:02.583257 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:02.583169    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:02.698712 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:02.698674    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:03.834872 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:03.834815    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:03.934783 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:03.934742    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:04.051755 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:04.051723    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:04.848066 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:04.848030    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:05.577366 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:05.577335    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"]
Sep 15 21:15:05.577607 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:05.577579    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw" podUID="98b8aab3-7b1d-4723-b152-ec7f53357a2b" containerName="maas-api" containerID="cri-o://4f18fe9f386a40679ac13378ffbd140fa68c6f253bf9ee747da3ecaed480ce16" gracePeriod=30
Sep 15 21:15:05.603153 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:05.603095    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:05.642104 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:05.641949    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7"]
Sep 15 21:15:05.643785 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:05.642210    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7" podUID="3c11383a-a64c-49e0-9b91-110607ed70be" containerName="payload-pre-processing" containerID="cri-o://8beb229bfce94d1f3e58b57582fab9ae52d1c83779c0c025211075bc14893a3e" gracePeriod=30
Sep 15 21:15:05.699204 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:05.699168    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:05.797869 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:05.797826    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:05.898363 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:05.898328    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:05.912475 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:05.912448    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"
Sep 15 21:15:05.939921 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:05.939897    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7"
Sep 15 21:15:06.029056 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.029010    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tggmb\" (UniqueName: \"kubernetes.io/projected/3c11383a-a64c-49e0-9b91-110607ed70be-kube-api-access-tggmb\") pod \"3c11383a-a64c-49e0-9b91-110607ed70be\" (UID: \"3c11383a-a64c-49e0-9b91-110607ed70be\") "
Sep 15 21:15:06.029258 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.029087    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c11383a-a64c-49e0-9b91-110607ed70be-plugins-config-volume\") pod \"3c11383a-a64c-49e0-9b91-110607ed70be\" (UID: \"3c11383a-a64c-49e0-9b91-110607ed70be\") "
Sep 15 21:15:06.029258 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.029124    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98b8aab3-7b1d-4723-b152-ec7f53357a2b-maas-api-tls\") pod \"98b8aab3-7b1d-4723-b152-ec7f53357a2b\" (UID: \"98b8aab3-7b1d-4723-b152-ec7f53357a2b\") "
Sep 15 21:15:06.029258 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.029169    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98b8aab3-7b1d-4723-b152-ec7f53357a2b-openshift-service-ca\") pod \"98b8aab3-7b1d-4723-b152-ec7f53357a2b\" (UID: \"98b8aab3-7b1d-4723-b152-ec7f53357a2b\") "
Sep 15 21:15:06.029258 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.029222    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98b8aab3-7b1d-4723-b152-ec7f53357a2b-metrics-certs\") pod \"98b8aab3-7b1d-4723-b152-ec7f53357a2b\" (UID: \"98b8aab3-7b1d-4723-b152-ec7f53357a2b\") "
Sep 15 21:15:06.029613 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.029268    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cnp6\" (UniqueName: \"kubernetes.io/projected/98b8aab3-7b1d-4723-b152-ec7f53357a2b-kube-api-access-2cnp6\") pod \"98b8aab3-7b1d-4723-b152-ec7f53357a2b\" (UID: \"98b8aab3-7b1d-4723-b152-ec7f53357a2b\") "
Sep 15 21:15:06.029935 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.029746    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98b8aab3-7b1d-4723-b152-ec7f53357a2b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "98b8aab3-7b1d-4723-b152-ec7f53357a2b" (UID: "98b8aab3-7b1d-4723-b152-ec7f53357a2b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:15:06.029935 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.029887    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c11383a-a64c-49e0-9b91-110607ed70be-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3c11383a-a64c-49e0-9b91-110607ed70be" (UID: "3c11383a-a64c-49e0-9b91-110607ed70be"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:15:06.031759 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.031730    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98b8aab3-7b1d-4723-b152-ec7f53357a2b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "98b8aab3-7b1d-4723-b152-ec7f53357a2b" (UID: "98b8aab3-7b1d-4723-b152-ec7f53357a2b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:15:06.031846 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.031813    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98b8aab3-7b1d-4723-b152-ec7f53357a2b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "98b8aab3-7b1d-4723-b152-ec7f53357a2b" (UID: "98b8aab3-7b1d-4723-b152-ec7f53357a2b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:15:06.032045 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.032032    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c11383a-a64c-49e0-9b91-110607ed70be-kube-api-access-tggmb" (OuterVolumeSpecName: "kube-api-access-tggmb") pod "3c11383a-a64c-49e0-9b91-110607ed70be" (UID: "3c11383a-a64c-49e0-9b91-110607ed70be"). InnerVolumeSpecName "kube-api-access-tggmb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:15:06.032083 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.032058    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98b8aab3-7b1d-4723-b152-ec7f53357a2b-kube-api-access-2cnp6" (OuterVolumeSpecName: "kube-api-access-2cnp6") pod "98b8aab3-7b1d-4723-b152-ec7f53357a2b" (UID: "98b8aab3-7b1d-4723-b152-ec7f53357a2b"). InnerVolumeSpecName "kube-api-access-2cnp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:15:06.131361 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.131096    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tggmb\" (UniqueName: \"kubernetes.io/projected/3c11383a-a64c-49e0-9b91-110607ed70be-kube-api-access-tggmb\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:15:06.131361 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.131132    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c11383a-a64c-49e0-9b91-110607ed70be-plugins-config-volume\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:15:06.131361 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.131146    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98b8aab3-7b1d-4723-b152-ec7f53357a2b-maas-api-tls\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:15:06.131361 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.131159    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98b8aab3-7b1d-4723-b152-ec7f53357a2b-openshift-service-ca\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:15:06.131361 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.131173    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98b8aab3-7b1d-4723-b152-ec7f53357a2b-metrics-certs\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:15:06.131361 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.131187    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cnp6\" (UniqueName: \"kubernetes.io/projected/98b8aab3-7b1d-4723-b152-ec7f53357a2b-kube-api-access-2cnp6\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:15:06.146578 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.146541    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:06.172665 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.172623    2564 generic.go:356] "Generic (PLEG): container finished" podID="98b8aab3-7b1d-4723-b152-ec7f53357a2b" containerID="4f18fe9f386a40679ac13378ffbd140fa68c6f253bf9ee747da3ecaed480ce16" exitCode=0
Sep 15 21:15:06.172852 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.172676    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"
Sep 15 21:15:06.172852 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.172717    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw" event={"ID":"98b8aab3-7b1d-4723-b152-ec7f53357a2b","Type":"ContainerDied","Data":"4f18fe9f386a40679ac13378ffbd140fa68c6f253bf9ee747da3ecaed480ce16"}
Sep 15 21:15:06.172852 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.172774    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw" event={"ID":"98b8aab3-7b1d-4723-b152-ec7f53357a2b","Type":"ContainerDied","Data":"3e48606358f303e54ca301d615ff5b6588412ba4f1bfec9ca71ec720bf8994f1"}
Sep 15 21:15:06.172852 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.172805    2564 scope.go:117] "RemoveContainer" containerID="4f18fe9f386a40679ac13378ffbd140fa68c6f253bf9ee747da3ecaed480ce16"
Sep 15 21:15:06.174264 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.174243    2564 generic.go:356] "Generic (PLEG): container finished" podID="3c11383a-a64c-49e0-9b91-110607ed70be" containerID="8beb229bfce94d1f3e58b57582fab9ae52d1c83779c0c025211075bc14893a3e" exitCode=0
Sep 15 21:15:06.174383 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.174281    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7"
Sep 15 21:15:06.174383 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.174324    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7" event={"ID":"3c11383a-a64c-49e0-9b91-110607ed70be","Type":"ContainerDied","Data":"8beb229bfce94d1f3e58b57582fab9ae52d1c83779c0c025211075bc14893a3e"}
Sep 15 21:15:06.174383 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.174376    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7" event={"ID":"3c11383a-a64c-49e0-9b91-110607ed70be","Type":"ContainerDied","Data":"cbb70cabcd7863b3f052c52ee774c72d61e82eee8279399638e3f5c0e81826f3"}
Sep 15 21:15:06.183031 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.183012    2564 scope.go:117] "RemoveContainer" containerID="4f18fe9f386a40679ac13378ffbd140fa68c6f253bf9ee747da3ecaed480ce16"
Sep 15 21:15:06.183392 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:15:06.183370    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f18fe9f386a40679ac13378ffbd140fa68c6f253bf9ee747da3ecaed480ce16\": container with ID starting with 4f18fe9f386a40679ac13378ffbd140fa68c6f253bf9ee747da3ecaed480ce16 not found: ID does not exist" containerID="4f18fe9f386a40679ac13378ffbd140fa68c6f253bf9ee747da3ecaed480ce16"
Sep 15 21:15:06.183520 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.183404    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f18fe9f386a40679ac13378ffbd140fa68c6f253bf9ee747da3ecaed480ce16"} err="failed to get container status \"4f18fe9f386a40679ac13378ffbd140fa68c6f253bf9ee747da3ecaed480ce16\": rpc error: code = NotFound desc = could not find container \"4f18fe9f386a40679ac13378ffbd140fa68c6f253bf9ee747da3ecaed480ce16\": container with ID starting with 4f18fe9f386a40679ac13378ffbd140fa68c6f253bf9ee747da3ecaed480ce16 not found: ID does not exist"
Sep 15 21:15:06.183520 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.183428    2564 scope.go:117] "RemoveContainer" containerID="8beb229bfce94d1f3e58b57582fab9ae52d1c83779c0c025211075bc14893a3e"
Sep 15 21:15:06.192219 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.192200    2564 scope.go:117] "RemoveContainer" containerID="8beb229bfce94d1f3e58b57582fab9ae52d1c83779c0c025211075bc14893a3e"
Sep 15 21:15:06.192533 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:15:06.192508    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8beb229bfce94d1f3e58b57582fab9ae52d1c83779c0c025211075bc14893a3e\": container with ID starting with 8beb229bfce94d1f3e58b57582fab9ae52d1c83779c0c025211075bc14893a3e not found: ID does not exist" containerID="8beb229bfce94d1f3e58b57582fab9ae52d1c83779c0c025211075bc14893a3e"
Sep 15 21:15:06.192630 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.192539    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8beb229bfce94d1f3e58b57582fab9ae52d1c83779c0c025211075bc14893a3e"} err="failed to get container status \"8beb229bfce94d1f3e58b57582fab9ae52d1c83779c0c025211075bc14893a3e\": rpc error: code = NotFound desc = could not find container \"8beb229bfce94d1f3e58b57582fab9ae52d1c83779c0c025211075bc14893a3e\": container with ID starting with 8beb229bfce94d1f3e58b57582fab9ae52d1c83779c0c025211075bc14893a3e not found: ID does not exist"
Sep 15 21:15:06.208850 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.208816    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"]
Sep 15 21:15:06.212727 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.212702    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-dca193-ffb957666-thstw"]
Sep 15 21:15:06.221668 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.221634    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:06.226129 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.226103    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7"]
Sep 15 21:15:06.229612 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:06.229591    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-dca193-68cd5995cd-pqlk7"]
Sep 15 21:15:07.069153 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:07.069120    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:07.906954 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:07.906915    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:07.974974 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:07.974935    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c11383a-a64c-49e0-9b91-110607ed70be" path="/var/lib/kubelet/pods/3c11383a-a64c-49e0-9b91-110607ed70be/volumes"
Sep 15 21:15:07.975352 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:07.975335    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98b8aab3-7b1d-4723-b152-ec7f53357a2b" path="/var/lib/kubelet/pods/98b8aab3-7b1d-4723-b152-ec7f53357a2b/volumes"
Sep 15 21:15:08.616560 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:08.616517    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:09.204587 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:09.204552    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:09.318551 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:09.318472    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:09.389059 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:09.389023    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:15:09.389232 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:09.389084    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:15:09.407174 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:09.407134    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:15:09.595929 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:09.595836    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:10.209120 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:10.209088    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:15:10.533172 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:10.533081    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:11.378728 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:11.378696    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:13.200165 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:13.200135    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:13.302242 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:13.302182    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:14.045476 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:14.045439    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:14.878955 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:14.878920    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:15.572217 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:15.572176    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:16.277058 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:16.277027    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:17.142223 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:17.142190    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:17.794902 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:17.794870    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:18.004387 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:18.004355    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:18.777371 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:18.777323    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:23.142333 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:23.142265    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:23.736605 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:23.736569    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:23.813205 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:23.813169    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:23.870504 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:23.870469    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:23.932045 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:23.932007    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:23.985120 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:23.985084    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:25.074333 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:25.074301    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:25.175335 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:25.175303    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:25.275157 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:25.275123    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:26.057377 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:26.057328    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:26.742876 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:26.742840    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:27.684101 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:27.684058    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:30.081339 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:30.081303    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:30.144682 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:30.144642    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:30.348113 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:30.348073    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:30.689775 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:30.689686    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:30.789670 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:30.789640    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:30.889924 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:30.889889    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:35.761449 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:35.761361    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:35.819086 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:35.819056    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:36.382848 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:36.382807    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:37.096968 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:37.096939    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:37.281967 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:37.281931    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:37.386883 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:37.386842    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:37.486226 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:37.486192    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:42.214544 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:42.214482    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:43.859175 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:43.859139    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:43.959529 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:43.959454    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:44.908763 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:44.908730    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:45.170603 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:45.170517    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:45.400784 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:45.400747    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:48.019388 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:48.019336    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:48.667319 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:48.667281    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:48.755692 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:48.755658    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:48.986891 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:48.986809    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:49.983760 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:49.983718    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:50.805710 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:50.805673    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:51.494847 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:51.494796    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:52.371353 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:52.371308    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:52.472382 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:52.472343    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:52.572559 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:52.572528    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:53.002277 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:53.002242    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:53.099329 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:53.099293    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:53.984778 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:53.984731    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:15:54.086931 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:15:54.086885    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:01.490267 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:01.490233    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:01.853847 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:01.853813    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:02.862376 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:02.862334    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:03.682575 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:03.682539    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:03.846387 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:03.846353    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:04.467892 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:04.467849    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:05.185264 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:05.185226    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:06.176016 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:06.175955    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:06.994458 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:06.994421    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:07.749945 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:07.749882    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:08.548759 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:08.548728    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:08.674906 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:08.674865    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:08.828986 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:08.828903    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:09.157084 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:09.157033    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:09.257030 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:09.256968    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:09.471807 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:09.471714    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:10.211141 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:10.211103    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:11.017917 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:11.017879    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:11.996802 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:11.996769    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:12.533835 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:12.533799    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:12.638526 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:12.638463    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:12.986333 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:12.986297    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:13.060041 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:13.059991    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:13.133346 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:13.133315    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:13.217873 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:13.217823    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:13.293724 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:13.293622    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:13.396414 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:13.396344    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:13.593045 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:13.592938    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:14.157919 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:14.157874    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:15.054680 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:15.054650    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:15.889097 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:15.889061    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:16.607570 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:16.607529    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:16.707295 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:16.707251    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:16.808279 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:16.808219    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:17.142242 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:17.142213    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:17.239787 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:17.239753    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:17.581563 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:17.581443    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:18.474119 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:18.474081    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:19.480329 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:19.480293    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:21.290752 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:21.290708    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:21.390080 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:21.390041    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:22.259049 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:22.259011    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:22.358848 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:22.358809    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:22.492674 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:22.492641    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:28.917190 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:28.917152    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:38.742242 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:38.742196    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:39.593444 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:39.593408    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:40.414544 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:40.414510    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:41.118948 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:41.118904    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:41.948706 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:41.948670    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:42.584925 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:42.584894    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:42.759477 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:42.759439    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:51.262966 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:51.262925    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:51.556889 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:51.556781    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:51.893734 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:51.893702    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:54.286180 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:54.286136    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:55.075897 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:55.075859    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:55.178319 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:55.178271    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:55.995110 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:55.995073    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:57.093248 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:57.093210    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:57.883251 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:57.883211    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:58.588187 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:58.588153    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:59.463848 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:59.463810    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"]
Sep 15 21:16:59.464475 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:59.464458    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98b8aab3-7b1d-4723-b152-ec7f53357a2b" containerName="maas-api"
Sep 15 21:16:59.464567 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:59.464479    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="98b8aab3-7b1d-4723-b152-ec7f53357a2b" containerName="maas-api"
Sep 15 21:16:59.464567 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:59.464522    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c11383a-a64c-49e0-9b91-110607ed70be" containerName="payload-pre-processing"
Sep 15 21:16:59.464567 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:59.464531    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c11383a-a64c-49e0-9b91-110607ed70be" containerName="payload-pre-processing"
Sep 15 21:16:59.464684 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:59.464628    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="98b8aab3-7b1d-4723-b152-ec7f53357a2b" containerName="maas-api"
Sep 15 21:16:59.464684 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:59.464641    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c11383a-a64c-49e0-9b91-110607ed70be" containerName="payload-pre-processing"
Sep 15 21:16:59.468472 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:59.468448    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"
Sep 15 21:16:59.471273 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:59.471251    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-56c6b012-serving-cert\""
Sep 15 21:16:59.480919 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:59.480883    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"]
Sep 15 21:16:59.501637 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:59.501601    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:59.553156 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:59.553123    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b4b628b-5962-423b-bfeb-5c63ec34caeb-openshift-service-ca\") pod \"maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8\" (UID: \"4b4b628b-5962-423b-bfeb-5c63ec34caeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"
Sep 15 21:16:59.553343 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:59.553203    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7vrg\" (UniqueName: \"kubernetes.io/projected/4b4b628b-5962-423b-bfeb-5c63ec34caeb-kube-api-access-n7vrg\") pod \"maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8\" (UID: \"4b4b628b-5962-423b-bfeb-5c63ec34caeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"
Sep 15 21:16:59.553343 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:59.553249    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b4b628b-5962-423b-bfeb-5c63ec34caeb-metrics-certs\") pod \"maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8\" (UID: \"4b4b628b-5962-423b-bfeb-5c63ec34caeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"
Sep 15 21:16:59.553343 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:59.553281    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b4b628b-5962-423b-bfeb-5c63ec34caeb-maas-api-tls\") pod \"maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8\" (UID: \"4b4b628b-5962-423b-bfeb-5c63ec34caeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"
Sep 15 21:16:59.602825 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:59.602790    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:16:59.654722 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:59.654670    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7vrg\" (UniqueName: \"kubernetes.io/projected/4b4b628b-5962-423b-bfeb-5c63ec34caeb-kube-api-access-n7vrg\") pod \"maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8\" (UID: \"4b4b628b-5962-423b-bfeb-5c63ec34caeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"
Sep 15 21:16:59.654912 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:59.654751    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b4b628b-5962-423b-bfeb-5c63ec34caeb-metrics-certs\") pod \"maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8\" (UID: \"4b4b628b-5962-423b-bfeb-5c63ec34caeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"
Sep 15 21:16:59.654912 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:59.654786    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b4b628b-5962-423b-bfeb-5c63ec34caeb-maas-api-tls\") pod \"maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8\" (UID: \"4b4b628b-5962-423b-bfeb-5c63ec34caeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"
Sep 15 21:16:59.654912 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:59.654875    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b4b628b-5962-423b-bfeb-5c63ec34caeb-openshift-service-ca\") pod \"maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8\" (UID: \"4b4b628b-5962-423b-bfeb-5c63ec34caeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"
Sep 15 21:16:59.655054 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:16:59.654967    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-serving-cert: secret "maas-api-e2e-mt-56c6b012-serving-cert" not found
Sep 15 21:16:59.655054 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:16:59.655039    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b4b628b-5962-423b-bfeb-5c63ec34caeb-maas-api-tls podName:4b4b628b-5962-423b-bfeb-5c63ec34caeb nodeName:}" failed. No retries permitted until 2026-09-15 21:17:00.155020571 +0000 UTC m=+1156.697183201 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4b4b628b-5962-423b-bfeb-5c63ec34caeb-maas-api-tls") pod "maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8" (UID: "4b4b628b-5962-423b-bfeb-5c63ec34caeb") : secret "maas-api-e2e-mt-56c6b012-serving-cert" not found
Sep 15 21:16:59.655649 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:59.655623    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b4b628b-5962-423b-bfeb-5c63ec34caeb-openshift-service-ca\") pod \"maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8\" (UID: \"4b4b628b-5962-423b-bfeb-5c63ec34caeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"
Sep 15 21:16:59.657453 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:59.657427    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b4b628b-5962-423b-bfeb-5c63ec34caeb-metrics-certs\") pod \"maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8\" (UID: \"4b4b628b-5962-423b-bfeb-5c63ec34caeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"
Sep 15 21:16:59.664134 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:16:59.664113    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7vrg\" (UniqueName: \"kubernetes.io/projected/4b4b628b-5962-423b-bfeb-5c63ec34caeb-kube-api-access-n7vrg\") pod \"maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8\" (UID: \"4b4b628b-5962-423b-bfeb-5c63ec34caeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"
Sep 15 21:17:00.160836 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:00.160790    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b4b628b-5962-423b-bfeb-5c63ec34caeb-maas-api-tls\") pod \"maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8\" (UID: \"4b4b628b-5962-423b-bfeb-5c63ec34caeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"
Sep 15 21:17:00.163541 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:00.163512    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b4b628b-5962-423b-bfeb-5c63ec34caeb-maas-api-tls\") pod \"maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8\" (UID: \"4b4b628b-5962-423b-bfeb-5c63ec34caeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"
Sep 15 21:17:00.386573 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:00.386520    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"
Sep 15 21:17:00.660547 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:00.660516    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"]
Sep 15 21:17:00.662335 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:17:00.662307    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4b628b_5962_423b_bfeb_5c63ec34caeb.slice/crio-69df5006870575ea19142d4633638fe1d32fe351d92ca0b84af831a4973a6473 WatchSource:0}: Error finding container 69df5006870575ea19142d4633638fe1d32fe351d92ca0b84af831a4973a6473: Status 404 returned error can't find the container with id 69df5006870575ea19142d4633638fe1d32fe351d92ca0b84af831a4973a6473
Sep 15 21:17:00.663721 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:00.663700    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 21:17:00.791234 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:00.791139    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8" event={"ID":"4b4b628b-5962-423b-bfeb-5c63ec34caeb","Type":"ContainerStarted","Data":"69df5006870575ea19142d4633638fe1d32fe351d92ca0b84af831a4973a6473"}
Sep 15 21:17:01.230764 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:01.230730    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:01.333293 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:01.333255    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:01.797988 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:01.797956    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8" event={"ID":"4b4b628b-5962-423b-bfeb-5c63ec34caeb","Type":"ContainerStarted","Data":"5fffc74e43fabf574224c7d3dfa308d8fb2117713fa4fcf39c88c7c6a3aa1573"}
Sep 15 21:17:01.798388 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:01.798086    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"
Sep 15 21:17:01.826819 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:01.826763    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8" podStartSLOduration=2.294803531 podStartE2EDuration="2.826747886s" totalImagesPullingTime="531.944355ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:16:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:17:01.824789865 +0000 UTC m=+1158.366952519" watchObservedRunningTime="2026-09-15 21:17:01.826747886 +0000 UTC m=+1158.368910520"
Sep 15 21:17:03.859610 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:03.859565    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:03.995107 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:03.995072    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:04.098295 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.098258    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:04.508443 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.508401    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"]
Sep 15 21:17:04.512110 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.512085    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:04.517473 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.517447    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-544890\"/\"distinct-w0-544890-kserve-self-signed-certs\""
Sep 15 21:17:04.526507 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.526460    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"]
Sep 15 21:17:04.605399 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.605361    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-tls-certs\") pod \"distinct-w0-544890-kserve-56489cd5cf-wthmg\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") " pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:04.605617 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.605430    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-tmp-dir\") pod \"distinct-w0-544890-kserve-56489cd5cf-wthmg\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") " pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:04.605617 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.605452    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-model-cache\") pod \"distinct-w0-544890-kserve-56489cd5cf-wthmg\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") " pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:04.605617 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.605474    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-dshm\") pod \"distinct-w0-544890-kserve-56489cd5cf-wthmg\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") " pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:04.605617 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.605541    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-home\") pod \"distinct-w0-544890-kserve-56489cd5cf-wthmg\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") " pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:04.605617 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.605563    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64462\" (UniqueName: \"kubernetes.io/projected/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-kube-api-access-64462\") pod \"distinct-w0-544890-kserve-56489cd5cf-wthmg\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") " pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:04.706996 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.706917    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-tls-certs\") pod \"distinct-w0-544890-kserve-56489cd5cf-wthmg\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") " pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:04.706996 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.706979    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-tmp-dir\") pod \"distinct-w0-544890-kserve-56489cd5cf-wthmg\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") " pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:04.706996 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.706998    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-model-cache\") pod \"distinct-w0-544890-kserve-56489cd5cf-wthmg\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") " pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:04.707296 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.707117    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-dshm\") pod \"distinct-w0-544890-kserve-56489cd5cf-wthmg\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") " pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:04.707296 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.707160    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-home\") pod \"distinct-w0-544890-kserve-56489cd5cf-wthmg\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") " pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:04.707296 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.707193    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64462\" (UniqueName: \"kubernetes.io/projected/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-kube-api-access-64462\") pod \"distinct-w0-544890-kserve-56489cd5cf-wthmg\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") " pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:04.707413 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.707365    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-model-cache\") pod \"distinct-w0-544890-kserve-56489cd5cf-wthmg\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") " pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:04.707477 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.707423    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-tmp-dir\") pod \"distinct-w0-544890-kserve-56489cd5cf-wthmg\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") " pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:04.707586 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.707544    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-home\") pod \"distinct-w0-544890-kserve-56489cd5cf-wthmg\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") " pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:04.709551 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.709531    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-dshm\") pod \"distinct-w0-544890-kserve-56489cd5cf-wthmg\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") " pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:04.709942 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.709923    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-tls-certs\") pod \"distinct-w0-544890-kserve-56489cd5cf-wthmg\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") " pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:04.718562 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.718536    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64462\" (UniqueName: \"kubernetes.io/projected/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-kube-api-access-64462\") pod \"distinct-w0-544890-kserve-56489cd5cf-wthmg\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") " pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:04.821609 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.821484    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:04.902888 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:04.902851    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:05.108604 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:05.108576    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"]
Sep 15 21:17:05.110711 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:17:05.110675    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod37bfe67c_8ca3_4ea1_916f_8012b9ec3195.slice/crio-c9d0d79ef1abf69b13afe64d726828359ca4b717e0cde3518c4b8259445017f6 WatchSource:0}: Error finding container c9d0d79ef1abf69b13afe64d726828359ca4b717e0cde3518c4b8259445017f6: Status 404 returned error can't find the container with id c9d0d79ef1abf69b13afe64d726828359ca4b717e0cde3518c4b8259445017f6
Sep 15 21:17:05.820749 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:05.820711    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg" event={"ID":"37bfe67c-8ca3-4ea1-916f-8012b9ec3195","Type":"ContainerStarted","Data":"f8c77b83856f1439774c439b5c19ea61aff299edf4c033dd39e15fd21f765d15"}
Sep 15 21:17:05.820749 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:05.820756    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg" event={"ID":"37bfe67c-8ca3-4ea1-916f-8012b9ec3195","Type":"ContainerStarted","Data":"c9d0d79ef1abf69b13afe64d726828359ca4b717e0cde3518c4b8259445017f6"}
Sep 15 21:17:05.846652 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:05.846586    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg" podStartSLOduration=1.8465677 podStartE2EDuration="1.8465677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:17:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:17:05.843637474 +0000 UTC m=+1162.385800109" watchObservedRunningTime="2026-09-15 21:17:05.8465677 +0000 UTC m=+1162.388730346"
Sep 15 21:17:07.809628 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:07.809595    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"
Sep 15 21:17:09.214896 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:09.214858    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:09.428268 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:09.428229    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:09.871262 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:09.871225    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:10.152136 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:10.152046    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:14.822629 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:14.822594    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:14.823069 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:14.822653    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:14.835557 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:14.835532    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:14.877523 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:14.877474    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:17:21.640328 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:21.640292    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:21.906428 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:21.906346    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:22.774390 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:22.774347    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:23.625986 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:23.625936    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:24.543501 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:24.543440    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:25.950333 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:25.950290    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:26.054797 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:26.054763    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:26.339321 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:26.339240    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:27.268033 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:27.267990    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:29.944539 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:29.944503    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"]
Sep 15 21:17:29.961075 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:29.961029    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:30.199693 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:30.199598    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:31.048196 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:31.048155    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:31.980357 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:31.980326    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:32.731808 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:32.731776    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:33.474448 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:33.474414    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:34.287592 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:34.287557    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:34.968115 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:34.968079    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:38.132120 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:38.132089    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:40.606230 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:40.606190    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"]
Sep 15 21:17:40.606770 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:40.606459    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk" podUID="6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3" containerName="maas-api" containerID="cri-o://3cbb9e171211939f2d93f110eef81dff4bcd86cf60a80da1cc6f160f917ab091" gracePeriod=30
Sep 15 21:17:40.637628 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:40.637592    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:40.653341 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:40.653297    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk" podUID="6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.46:8443/health\": dial tcp 10.132.0.46:8443: connect: connection refused"
Sep 15 21:17:40.711178 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:40.711138    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:40.778555 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:40.778509    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:40.857244 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:40.857150    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:40.879245 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:40.879220    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"
Sep 15 21:17:40.917886 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:40.917847    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:40.971162 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:40.971131    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-maas-api-tls\") pod \"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3\" (UID: \"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3\") "
Sep 15 21:17:40.971346 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:40.971179    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fv4k\" (UniqueName: \"kubernetes.io/projected/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-kube-api-access-8fv4k\") pod \"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3\" (UID: \"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3\") "
Sep 15 21:17:40.971346 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:40.971241    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-openshift-service-ca\") pod \"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3\" (UID: \"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3\") "
Sep 15 21:17:40.971346 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:40.971295    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-metrics-certs\") pod \"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3\" (UID: \"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3\") "
Sep 15 21:17:40.971926 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:40.971897    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3" (UID: "6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:17:40.973714 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:40.973683    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3" (UID: "6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:17:40.973714 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:40.973696    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-kube-api-access-8fv4k" (OuterVolumeSpecName: "kube-api-access-8fv4k") pod "6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3" (UID: "6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3"). InnerVolumeSpecName "kube-api-access-8fv4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:17:40.974253 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:40.974231    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3" (UID: "6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:17:41.021746 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:41.021714    2564 generic.go:356] "Generic (PLEG): container finished" podID="6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3" containerID="3cbb9e171211939f2d93f110eef81dff4bcd86cf60a80da1cc6f160f917ab091" exitCode=0
Sep 15 21:17:41.021918 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:41.021771    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk" event={"ID":"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3","Type":"ContainerDied","Data":"3cbb9e171211939f2d93f110eef81dff4bcd86cf60a80da1cc6f160f917ab091"}
Sep 15 21:17:41.021918 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:41.021817    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk" event={"ID":"6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3","Type":"ContainerDied","Data":"b823653b7c0389e56163d91ded011dd4d8350b27f5aa8658752374c71d45258b"}
Sep 15 21:17:41.021918 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:41.021839    2564 scope.go:117] "RemoveContainer" containerID="3cbb9e171211939f2d93f110eef81dff4bcd86cf60a80da1cc6f160f917ab091"
Sep 15 21:17:41.021918 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:41.021773    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"
Sep 15 21:17:41.032423 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:41.032397    2564 scope.go:117] "RemoveContainer" containerID="3cbb9e171211939f2d93f110eef81dff4bcd86cf60a80da1cc6f160f917ab091"
Sep 15 21:17:41.032779 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:17:41.032758    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cbb9e171211939f2d93f110eef81dff4bcd86cf60a80da1cc6f160f917ab091\": container with ID starting with 3cbb9e171211939f2d93f110eef81dff4bcd86cf60a80da1cc6f160f917ab091 not found: ID does not exist" containerID="3cbb9e171211939f2d93f110eef81dff4bcd86cf60a80da1cc6f160f917ab091"
Sep 15 21:17:41.032840 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:41.032790    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cbb9e171211939f2d93f110eef81dff4bcd86cf60a80da1cc6f160f917ab091"} err="failed to get container status \"3cbb9e171211939f2d93f110eef81dff4bcd86cf60a80da1cc6f160f917ab091\": rpc error: code = NotFound desc = could not find container \"3cbb9e171211939f2d93f110eef81dff4bcd86cf60a80da1cc6f160f917ab091\": container with ID starting with 3cbb9e171211939f2d93f110eef81dff4bcd86cf60a80da1cc6f160f917ab091 not found: ID does not exist"
Sep 15 21:17:41.044523 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:41.044463    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"]
Sep 15 21:17:41.048461 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:41.048436    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0df719-796f76c855-jg8xk"]
Sep 15 21:17:41.072927 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:41.072892    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-openshift-service-ca\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:17:41.072927 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:41.072925    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-metrics-certs\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:17:41.073079 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:41.072944    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-maas-api-tls\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:17:41.073079 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:41.072956    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fv4k\" (UniqueName: \"kubernetes.io/projected/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3-kube-api-access-8fv4k\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:17:41.211432 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:41.211337    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:41.311780 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:41.311744    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:41.980087 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:41.980038    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3" path="/var/lib/kubelet/pods/6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3/volumes"
Sep 15 21:17:42.671671 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:42.671622    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:42.882473 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:42.882420    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:43.789765 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:43.789733    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:43.981075 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:43.981041    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-7t46m_c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf/console-operator/1.log"
Sep 15 21:17:43.984331 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:43.984307    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4c7k_fc205950-caf0-4c47-84ce-a8fc54a1afb2/ovn-acl-logging/0.log"
Sep 15 21:17:43.984915 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:43.984898    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-7t46m_c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf/console-operator/1.log"
Sep 15 21:17:43.988149 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:43.988131    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4c7k_fc205950-caf0-4c47-84ce-a8fc54a1afb2/ovn-acl-logging/0.log"
Sep 15 21:17:44.511641 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:44.511603    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:44.968289 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:44.968252    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7" podUID="1034c0f8-2cd5-42e5-889d-eb78749d96da" containerName="main" containerID="cri-o://42174cc0830ba231a70691254bca159e313fc93d6aac78ec2cb8fb7a0965167b" gracePeriod=45
Sep 15 21:17:45.234290 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:45.234221    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:17:45.362479 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:45.362440    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:45.417477 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:45.417438    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-dshm\") pod \"1034c0f8-2cd5-42e5-889d-eb78749d96da\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") "
Sep 15 21:17:45.417477 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:45.417509    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-model-cache\") pod \"1034c0f8-2cd5-42e5-889d-eb78749d96da\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") "
Sep 15 21:17:45.417730 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:45.417540    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zw65q\" (UniqueName: \"kubernetes.io/projected/1034c0f8-2cd5-42e5-889d-eb78749d96da-kube-api-access-zw65q\") pod \"1034c0f8-2cd5-42e5-889d-eb78749d96da\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") "
Sep 15 21:17:45.417730 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:45.417599    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-tmp-dir\") pod \"1034c0f8-2cd5-42e5-889d-eb78749d96da\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") "
Sep 15 21:17:45.417730 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:45.417628    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1034c0f8-2cd5-42e5-889d-eb78749d96da-tls-certs\") pod \"1034c0f8-2cd5-42e5-889d-eb78749d96da\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") "
Sep 15 21:17:45.417894 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:45.417763    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-home\") pod \"1034c0f8-2cd5-42e5-889d-eb78749d96da\" (UID: \"1034c0f8-2cd5-42e5-889d-eb78749d96da\") "
Sep 15 21:17:45.417894 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:45.417864    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-model-cache" (OuterVolumeSpecName: "model-cache") pod "1034c0f8-2cd5-42e5-889d-eb78749d96da" (UID: "1034c0f8-2cd5-42e5-889d-eb78749d96da"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:17:45.417894 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:45.417879    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1034c0f8-2cd5-42e5-889d-eb78749d96da" (UID: "1034c0f8-2cd5-42e5-889d-eb78749d96da"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:17:45.418044 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:45.417963    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-home" (OuterVolumeSpecName: "home") pod "1034c0f8-2cd5-42e5-889d-eb78749d96da" (UID: "1034c0f8-2cd5-42e5-889d-eb78749d96da"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:17:45.418369 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:45.418343    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-model-cache\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:17:45.418524 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:45.418372    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-tmp-dir\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:17:45.418524 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:45.418386    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-home\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:17:45.420027 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:45.419999    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1034c0f8-2cd5-42e5-889d-eb78749d96da-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1034c0f8-2cd5-42e5-889d-eb78749d96da" (UID: "1034c0f8-2cd5-42e5-889d-eb78749d96da"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:17:45.420027 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:45.420018    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-dshm" (OuterVolumeSpecName: "dshm") pod "1034c0f8-2cd5-42e5-889d-eb78749d96da" (UID: "1034c0f8-2cd5-42e5-889d-eb78749d96da"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:17:45.420145 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:45.420036    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1034c0f8-2cd5-42e5-889d-eb78749d96da-kube-api-access-zw65q" (OuterVolumeSpecName: "kube-api-access-zw65q") pod "1034c0f8-2cd5-42e5-889d-eb78749d96da" (UID: "1034c0f8-2cd5-42e5-889d-eb78749d96da"). InnerVolumeSpecName "kube-api-access-zw65q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:17:45.520021 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:45.519926    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1034c0f8-2cd5-42e5-889d-eb78749d96da-dshm\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:17:45.520021 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:45.519963    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zw65q\" (UniqueName: \"kubernetes.io/projected/1034c0f8-2cd5-42e5-889d-eb78749d96da-kube-api-access-zw65q\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:17:45.520021 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:45.519975    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1034c0f8-2cd5-42e5-889d-eb78749d96da-tls-certs\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:17:46.050395 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:46.050359    2564 generic.go:356] "Generic (PLEG): container finished" podID="1034c0f8-2cd5-42e5-889d-eb78749d96da" containerID="42174cc0830ba231a70691254bca159e313fc93d6aac78ec2cb8fb7a0965167b" exitCode=0
Sep 15 21:17:46.050887 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:46.050418    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"
Sep 15 21:17:46.050887 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:46.050426    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7" event={"ID":"1034c0f8-2cd5-42e5-889d-eb78749d96da","Type":"ContainerDied","Data":"42174cc0830ba231a70691254bca159e313fc93d6aac78ec2cb8fb7a0965167b"}
Sep 15 21:17:46.050887 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:46.050459    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7" event={"ID":"1034c0f8-2cd5-42e5-889d-eb78749d96da","Type":"ContainerDied","Data":"d8cbd937cab565f45a29da9262c8112deda6a19ff7a1f9a5de5269fd77904601"}
Sep 15 21:17:46.050887 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:46.050475    2564 scope.go:117] "RemoveContainer" containerID="42174cc0830ba231a70691254bca159e313fc93d6aac78ec2cb8fb7a0965167b"
Sep 15 21:17:46.059840 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:46.059815    2564 scope.go:117] "RemoveContainer" containerID="42174cc0830ba231a70691254bca159e313fc93d6aac78ec2cb8fb7a0965167b"
Sep 15 21:17:46.060128 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:17:46.060107    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42174cc0830ba231a70691254bca159e313fc93d6aac78ec2cb8fb7a0965167b\": container with ID starting with 42174cc0830ba231a70691254bca159e313fc93d6aac78ec2cb8fb7a0965167b not found: ID does not exist" containerID="42174cc0830ba231a70691254bca159e313fc93d6aac78ec2cb8fb7a0965167b"
Sep 15 21:17:46.060199 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:46.060136    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42174cc0830ba231a70691254bca159e313fc93d6aac78ec2cb8fb7a0965167b"} err="failed to get container status \"42174cc0830ba231a70691254bca159e313fc93d6aac78ec2cb8fb7a0965167b\": rpc error: code = NotFound desc = could not find container \"42174cc0830ba231a70691254bca159e313fc93d6aac78ec2cb8fb7a0965167b\": container with ID starting with 42174cc0830ba231a70691254bca159e313fc93d6aac78ec2cb8fb7a0965167b not found: ID does not exist"
Sep 15 21:17:46.070322 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:46.070292    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"]
Sep 15 21:17:46.074374 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:46.074347    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-253ca1/test-model-253ca1-kserve-684d8bfc9-pnmb7"]
Sep 15 21:17:47.292193 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:47.292155    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:47.628175 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:47.628134    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:47.696705 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:47.696665    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:47.910552 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:47.910427    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:47.973929 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:47.973899    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1034c0f8-2cd5-42e5-889d-eb78749d96da" path="/var/lib/kubelet/pods/1034c0f8-2cd5-42e5-889d-eb78749d96da/volumes"
Sep 15 21:17:48.457328 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:48.457295    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:49.237065 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:49.237017    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:50.079500 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:50.079337    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:50.783701 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:50.783650    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:51.580898 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:51.580867    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:52.276928 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:52.276863    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:52.979130 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:52.979085    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:58.745810 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:58.745774    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"]
Sep 15 21:17:58.746223 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:58.746058    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7" podUID="0d03e0f2-93b3-497a-99db-d4c33188e293" containerName="maas-api" containerID="cri-o://b7f3681b3481251682fbb54714edbeaf732395dd8fa2ab19b4b90c2ff4c3d524" gracePeriod=30
Sep 15 21:17:58.804971 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:58.804931    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv"]
Sep 15 21:17:58.805464 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:58.805408    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv" podUID="b67c890d-216f-4e2f-bdd7-5b587e49ccb1" containerName="payload-processing" containerID="cri-o://c3d2f95326e772eaed8b4d7e74b3c3e59d0900189df747d749a91c3020256ccf" gracePeriod=30
Sep 15 21:17:58.816614 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:58.816537    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:58.913304 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:58.913278    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:58.996010 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:58.995925    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:59.057354 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.057324    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"
Sep 15 21:17:59.060004 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.059976    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:59.108050 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.108025    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv"
Sep 15 21:17:59.120695 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.120667    2564 generic.go:356] "Generic (PLEG): container finished" podID="b67c890d-216f-4e2f-bdd7-5b587e49ccb1" containerID="c3d2f95326e772eaed8b4d7e74b3c3e59d0900189df747d749a91c3020256ccf" exitCode=0
Sep 15 21:17:59.120884 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.120729    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv"
Sep 15 21:17:59.120884 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.120758    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv" event={"ID":"b67c890d-216f-4e2f-bdd7-5b587e49ccb1","Type":"ContainerDied","Data":"c3d2f95326e772eaed8b4d7e74b3c3e59d0900189df747d749a91c3020256ccf"}
Sep 15 21:17:59.120884 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.120815    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv" event={"ID":"b67c890d-216f-4e2f-bdd7-5b587e49ccb1","Type":"ContainerDied","Data":"5d150a837be15b8791b6164b7951507924d6b72fd2a318c58b4488704c318869"}
Sep 15 21:17:59.120884 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.120841    2564 scope.go:117] "RemoveContainer" containerID="c3d2f95326e772eaed8b4d7e74b3c3e59d0900189df747d749a91c3020256ccf"
Sep 15 21:17:59.122600 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.122543    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7" event={"ID":"0d03e0f2-93b3-497a-99db-d4c33188e293","Type":"ContainerDied","Data":"b7f3681b3481251682fbb54714edbeaf732395dd8fa2ab19b4b90c2ff4c3d524"}
Sep 15 21:17:59.122600 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.122585    2564 generic.go:356] "Generic (PLEG): container finished" podID="0d03e0f2-93b3-497a-99db-d4c33188e293" containerID="b7f3681b3481251682fbb54714edbeaf732395dd8fa2ab19b4b90c2ff4c3d524" exitCode=0
Sep 15 21:17:59.122769 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.122560    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"
Sep 15 21:17:59.122769 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.122719    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7" event={"ID":"0d03e0f2-93b3-497a-99db-d4c33188e293","Type":"ContainerDied","Data":"98a3f09843688e69ddcafd8a33a1b0e2a9d931bddfc7027acbc577d4301ec61d"}
Sep 15 21:17:59.131870 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.131851    2564 scope.go:117] "RemoveContainer" containerID="c3d2f95326e772eaed8b4d7e74b3c3e59d0900189df747d749a91c3020256ccf"
Sep 15 21:17:59.132275 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:17:59.132248    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3d2f95326e772eaed8b4d7e74b3c3e59d0900189df747d749a91c3020256ccf\": container with ID starting with c3d2f95326e772eaed8b4d7e74b3c3e59d0900189df747d749a91c3020256ccf not found: ID does not exist" containerID="c3d2f95326e772eaed8b4d7e74b3c3e59d0900189df747d749a91c3020256ccf"
Sep 15 21:17:59.132374 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.132287    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3d2f95326e772eaed8b4d7e74b3c3e59d0900189df747d749a91c3020256ccf"} err="failed to get container status \"c3d2f95326e772eaed8b4d7e74b3c3e59d0900189df747d749a91c3020256ccf\": rpc error: code = NotFound desc = could not find container \"c3d2f95326e772eaed8b4d7e74b3c3e59d0900189df747d749a91c3020256ccf\": container with ID starting with c3d2f95326e772eaed8b4d7e74b3c3e59d0900189df747d749a91c3020256ccf not found: ID does not exist"
Sep 15 21:17:59.132374 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.132320    2564 scope.go:117] "RemoveContainer" containerID="b7f3681b3481251682fbb54714edbeaf732395dd8fa2ab19b4b90c2ff4c3d524"
Sep 15 21:17:59.143982 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.143953    2564 scope.go:117] "RemoveContainer" containerID="b7f3681b3481251682fbb54714edbeaf732395dd8fa2ab19b4b90c2ff4c3d524"
Sep 15 21:17:59.144181 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.144160    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qh5h6\" (UniqueName: \"kubernetes.io/projected/0d03e0f2-93b3-497a-99db-d4c33188e293-kube-api-access-qh5h6\") pod \"0d03e0f2-93b3-497a-99db-d4c33188e293\" (UID: \"0d03e0f2-93b3-497a-99db-d4c33188e293\") "
Sep 15 21:17:59.144254 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.144204    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b67c890d-216f-4e2f-bdd7-5b587e49ccb1-plugins-config-volume\") pod \"b67c890d-216f-4e2f-bdd7-5b587e49ccb1\" (UID: \"b67c890d-216f-4e2f-bdd7-5b587e49ccb1\") "
Sep 15 21:17:59.144254 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.144234    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d03e0f2-93b3-497a-99db-d4c33188e293-metrics-certs\") pod \"0d03e0f2-93b3-497a-99db-d4c33188e293\" (UID: \"0d03e0f2-93b3-497a-99db-d4c33188e293\") "
Sep 15 21:17:59.144372 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.144265    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k47gn\" (UniqueName: \"kubernetes.io/projected/b67c890d-216f-4e2f-bdd7-5b587e49ccb1-kube-api-access-k47gn\") pod \"b67c890d-216f-4e2f-bdd7-5b587e49ccb1\" (UID: \"b67c890d-216f-4e2f-bdd7-5b587e49ccb1\") "
Sep 15 21:17:59.144372 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.144356    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d03e0f2-93b3-497a-99db-d4c33188e293-maas-api-tls\") pod \"0d03e0f2-93b3-497a-99db-d4c33188e293\" (UID: \"0d03e0f2-93b3-497a-99db-d4c33188e293\") "
Sep 15 21:17:59.144481 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.144395    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d03e0f2-93b3-497a-99db-d4c33188e293-openshift-service-ca\") pod \"0d03e0f2-93b3-497a-99db-d4c33188e293\" (UID: \"0d03e0f2-93b3-497a-99db-d4c33188e293\") "
Sep 15 21:17:59.144727 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.144698    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b67c890d-216f-4e2f-bdd7-5b587e49ccb1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b67c890d-216f-4e2f-bdd7-5b587e49ccb1" (UID: "b67c890d-216f-4e2f-bdd7-5b587e49ccb1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:17:59.144823 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.144799    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b67c890d-216f-4e2f-bdd7-5b587e49ccb1-plugins-config-volume\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:17:59.145156 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.145129    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d03e0f2-93b3-497a-99db-d4c33188e293-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0d03e0f2-93b3-497a-99db-d4c33188e293" (UID: "0d03e0f2-93b3-497a-99db-d4c33188e293"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:17:59.145246 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:17:59.145197    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7f3681b3481251682fbb54714edbeaf732395dd8fa2ab19b4b90c2ff4c3d524\": container with ID starting with b7f3681b3481251682fbb54714edbeaf732395dd8fa2ab19b4b90c2ff4c3d524 not found: ID does not exist" containerID="b7f3681b3481251682fbb54714edbeaf732395dd8fa2ab19b4b90c2ff4c3d524"
Sep 15 21:17:59.145316 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.145237    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7f3681b3481251682fbb54714edbeaf732395dd8fa2ab19b4b90c2ff4c3d524"} err="failed to get container status \"b7f3681b3481251682fbb54714edbeaf732395dd8fa2ab19b4b90c2ff4c3d524\": rpc error: code = NotFound desc = could not find container \"b7f3681b3481251682fbb54714edbeaf732395dd8fa2ab19b4b90c2ff4c3d524\": container with ID starting with b7f3681b3481251682fbb54714edbeaf732395dd8fa2ab19b4b90c2ff4c3d524 not found: ID does not exist"
Sep 15 21:17:59.146933 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.146908    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d03e0f2-93b3-497a-99db-d4c33188e293-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0d03e0f2-93b3-497a-99db-d4c33188e293" (UID: "0d03e0f2-93b3-497a-99db-d4c33188e293"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:17:59.147060 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.147038    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d03e0f2-93b3-497a-99db-d4c33188e293-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0d03e0f2-93b3-497a-99db-d4c33188e293" (UID: "0d03e0f2-93b3-497a-99db-d4c33188e293"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:17:59.147426 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.147398    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d03e0f2-93b3-497a-99db-d4c33188e293-kube-api-access-qh5h6" (OuterVolumeSpecName: "kube-api-access-qh5h6") pod "0d03e0f2-93b3-497a-99db-d4c33188e293" (UID: "0d03e0f2-93b3-497a-99db-d4c33188e293"). InnerVolumeSpecName "kube-api-access-qh5h6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:17:59.147426 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.147412    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b67c890d-216f-4e2f-bdd7-5b587e49ccb1-kube-api-access-k47gn" (OuterVolumeSpecName: "kube-api-access-k47gn") pod "b67c890d-216f-4e2f-bdd7-5b587e49ccb1" (UID: "b67c890d-216f-4e2f-bdd7-5b587e49ccb1"). InnerVolumeSpecName "kube-api-access-k47gn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:17:59.246197 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.246076    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d03e0f2-93b3-497a-99db-d4c33188e293-maas-api-tls\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:17:59.246197 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.246126    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d03e0f2-93b3-497a-99db-d4c33188e293-openshift-service-ca\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:17:59.246197 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.246141    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qh5h6\" (UniqueName: \"kubernetes.io/projected/0d03e0f2-93b3-497a-99db-d4c33188e293-kube-api-access-qh5h6\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:17:59.246197 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.246153    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d03e0f2-93b3-497a-99db-d4c33188e293-metrics-certs\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:17:59.246197 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.246165    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k47gn\" (UniqueName: \"kubernetes.io/projected/b67c890d-216f-4e2f-bdd7-5b587e49ccb1-kube-api-access-k47gn\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:17:59.358848 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.358813    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:59.450222 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.450181    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv"]
Sep 15 21:17:59.454113 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.454086    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-253ca1-78f58d687c-96qsv"]
Sep 15 21:17:59.464790 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.464764    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"]
Sep 15 21:17:59.470942 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.470914    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-253ca1-76c45798c4-qfht7"]
Sep 15 21:17:59.475665 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.475619    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:17:59.973912 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.973882    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d03e0f2-93b3-497a-99db-d4c33188e293" path="/var/lib/kubelet/pods/0d03e0f2-93b3-497a-99db-d4c33188e293/volumes"
Sep 15 21:17:59.974285 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:17:59.974253    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b67c890d-216f-4e2f-bdd7-5b587e49ccb1" path="/var/lib/kubelet/pods/b67c890d-216f-4e2f-bdd7-5b587e49ccb1/volumes"
Sep 15 21:18:00.191368 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:00.191336    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:01.085455 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:01.085418    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:01.823006 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:01.822976    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:02.607430 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:02.607393    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:05.272441 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:05.272406    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:05.365979 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:05.365945    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:05.615906 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:05.615875    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:06.087153 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:06.087055    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:06.391174 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:06.391140    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:06.656551 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:06.656453    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:08.858085 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:08.858029    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:09.469221 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:09.469174    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:09.574675 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:09.574637    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:09.669842 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:09.669810    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:10.809508 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:10.809440    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:11.657710 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:11.657664    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:12.493205 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:12.493168    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:13.298770 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:13.298734    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:13.399567 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:13.399524    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:13.658825 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:13.658773    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:13.850454 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:13.850414    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:13.946287 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:13.946200    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:13.953798 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:13.953770    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66"]
Sep 15 21:18:13.954200 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:13.954187    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d03e0f2-93b3-497a-99db-d4c33188e293" containerName="maas-api"
Sep 15 21:18:13.954262 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:13.954202    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d03e0f2-93b3-497a-99db-d4c33188e293" containerName="maas-api"
Sep 15 21:18:13.954262 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:13.954216    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1034c0f8-2cd5-42e5-889d-eb78749d96da" containerName="main"
Sep 15 21:18:13.954262 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:13.954222    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1034c0f8-2cd5-42e5-889d-eb78749d96da" containerName="main"
Sep 15 21:18:13.954262 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:13.954243    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b67c890d-216f-4e2f-bdd7-5b587e49ccb1" containerName="payload-processing"
Sep 15 21:18:13.954262 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:13.954248    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b67c890d-216f-4e2f-bdd7-5b587e49ccb1" containerName="payload-processing"
Sep 15 21:18:13.954262 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:13.954258    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3" containerName="maas-api"
Sep 15 21:18:13.954262 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:13.954263    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3" containerName="maas-api"
Sep 15 21:18:13.954530 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:13.954324    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b67c890d-216f-4e2f-bdd7-5b587e49ccb1" containerName="payload-processing"
Sep 15 21:18:13.954530 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:13.954333    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d03e0f2-93b3-497a-99db-d4c33188e293" containerName="maas-api"
Sep 15 21:18:13.954530 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:13.954341    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1034c0f8-2cd5-42e5-889d-eb78749d96da" containerName="main"
Sep 15 21:18:13.954530 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:13.954350    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c99a9fa-0a11-4e5d-8e1d-1e02e46cf7f3" containerName="maas-api"
Sep 15 21:18:13.957557 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:13.957540    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66"
Sep 15 21:18:13.960353 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:13.960329    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-2f34ef\""
Sep 15 21:18:13.960519 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:13.960434    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-2f34ef-dockercfg-z7g4n\""
Sep 15 21:18:13.980853 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:13.980815    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66"]
Sep 15 21:18:14.086851 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:14.086813    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20a0b584-3f13-44d5-bce5-87e6f46927db-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66\" (UID: \"20a0b584-3f13-44d5-bce5-87e6f46927db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66"
Sep 15 21:18:14.087027 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:14.086927    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgtmw\" (UniqueName: \"kubernetes.io/projected/20a0b584-3f13-44d5-bce5-87e6f46927db-kube-api-access-dgtmw\") pod \"payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66\" (UID: \"20a0b584-3f13-44d5-bce5-87e6f46927db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66"
Sep 15 21:18:14.188300 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:14.188263    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgtmw\" (UniqueName: \"kubernetes.io/projected/20a0b584-3f13-44d5-bce5-87e6f46927db-kube-api-access-dgtmw\") pod \"payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66\" (UID: \"20a0b584-3f13-44d5-bce5-87e6f46927db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66"
Sep 15 21:18:14.188525 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:14.188383    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20a0b584-3f13-44d5-bce5-87e6f46927db-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66\" (UID: \"20a0b584-3f13-44d5-bce5-87e6f46927db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66"
Sep 15 21:18:14.189094 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:14.189074    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20a0b584-3f13-44d5-bce5-87e6f46927db-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66\" (UID: \"20a0b584-3f13-44d5-bce5-87e6f46927db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66"
Sep 15 21:18:14.197891 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:14.197816    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgtmw\" (UniqueName: \"kubernetes.io/projected/20a0b584-3f13-44d5-bce5-87e6f46927db-kube-api-access-dgtmw\") pod \"payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66\" (UID: \"20a0b584-3f13-44d5-bce5-87e6f46927db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66"
Sep 15 21:18:14.267652 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:14.267614    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66"
Sep 15 21:18:14.544854 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:14.544830    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66"]
Sep 15 21:18:14.547414 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:18:14.547389    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20a0b584_3f13_44d5_bce5_87e6f46927db.slice/crio-f04fdea7f24950962f536ed62e0cffe05d0902c41f4312f632e9340432ae8e47 WatchSource:0}: Error finding container f04fdea7f24950962f536ed62e0cffe05d0902c41f4312f632e9340432ae8e47: Status 404 returned error can't find the container with id f04fdea7f24950962f536ed62e0cffe05d0902c41f4312f632e9340432ae8e47
Sep 15 21:18:15.210044 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:15.210006    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66" event={"ID":"20a0b584-3f13-44d5-bce5-87e6f46927db","Type":"ContainerStarted","Data":"d3a090ad15b37413c985f16e24ee5c17a49db56d18813e1cd2142eac0d456e90"}
Sep 15 21:18:15.210646 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:15.210061    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66" event={"ID":"20a0b584-3f13-44d5-bce5-87e6f46927db","Type":"ContainerStarted","Data":"f04fdea7f24950962f536ed62e0cffe05d0902c41f4312f632e9340432ae8e47"}
Sep 15 21:18:15.210646 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:15.210128    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66"
Sep 15 21:18:15.253437 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:15.253381    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66" podStartSLOduration=2.25336301 podStartE2EDuration="2.25336301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:18:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:18:15.251250548 +0000 UTC m=+1231.793413196" watchObservedRunningTime="2026-09-15 21:18:15.25336301 +0000 UTC m=+1231.795525645"
Sep 15 21:18:22.425628 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:22.425584    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:26.219967 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:26.219929    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66"
Sep 15 21:18:34.305161 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.305129    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"]
Sep 15 21:18:34.309056 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.309026    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:34.312306 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.312274    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-544890\"/\"distinct-2-w0-544890-kserve-self-signed-certs\""
Sep 15 21:18:34.325377 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.325344    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"]
Sep 15 21:18:34.357221 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.357183    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:34.390288 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.390250    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-dshm\") pod \"distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") " pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:34.390288 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.390291    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-tmp-dir\") pod \"distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") " pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:34.390514 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.390373    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/784be271-ba64-4757-9c89-cc238f0e3f22-tls-certs\") pod \"distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") " pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:34.390514 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.390402    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzjsw\" (UniqueName: \"kubernetes.io/projected/784be271-ba64-4757-9c89-cc238f0e3f22-kube-api-access-qzjsw\") pod \"distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") " pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:34.390514 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.390444    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-home\") pod \"distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") " pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:34.390514 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.390483    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-model-cache\") pod \"distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") " pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:34.491269 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.491237    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-tmp-dir\") pod \"distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") " pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:34.491473 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.491296    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/784be271-ba64-4757-9c89-cc238f0e3f22-tls-certs\") pod \"distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") " pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:34.491473 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.491322    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzjsw\" (UniqueName: \"kubernetes.io/projected/784be271-ba64-4757-9c89-cc238f0e3f22-kube-api-access-qzjsw\") pod \"distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") " pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:34.491473 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.491382    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-home\") pod \"distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") " pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:34.491473 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.491415    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-model-cache\") pod \"distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") " pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:34.491756 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.491480    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-dshm\") pod \"distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") " pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:34.491817 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.491761    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-tmp-dir\") pod \"distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") " pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:34.491874 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.491805    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-home\") pod \"distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") " pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:34.491874 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.491847    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-model-cache\") pod \"distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") " pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:34.493899 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.493866    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-dshm\") pod \"distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") " pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:34.494036 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.494019    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/784be271-ba64-4757-9c89-cc238f0e3f22-tls-certs\") pod \"distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") " pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:34.499999 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.499977    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzjsw\" (UniqueName: \"kubernetes.io/projected/784be271-ba64-4757-9c89-cc238f0e3f22-kube-api-access-qzjsw\") pod \"distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") " pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:34.619200 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.619165    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:34.892851 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:34.892827    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"]
Sep 15 21:18:34.895936 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:18:34.895554    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod784be271_ba64_4757_9c89_cc238f0e3f22.slice/crio-50df71157557654fbc607487f90ca69fed8c2498f6858f15ec1fbdb02cec79df WatchSource:0}: Error finding container 50df71157557654fbc607487f90ca69fed8c2498f6858f15ec1fbdb02cec79df: Status 404 returned error can't find the container with id 50df71157557654fbc607487f90ca69fed8c2498f6858f15ec1fbdb02cec79df
Sep 15 21:18:35.320650 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:35.320549    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g" event={"ID":"784be271-ba64-4757-9c89-cc238f0e3f22","Type":"ContainerStarted","Data":"f3e46b0c9a5346172d5dacba28fe9f7a08c98b8dc617acf58eba15441cea7d29"}
Sep 15 21:18:35.320650 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:35.320600    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g" event={"ID":"784be271-ba64-4757-9c89-cc238f0e3f22","Type":"ContainerStarted","Data":"50df71157557654fbc607487f90ca69fed8c2498f6858f15ec1fbdb02cec79df"}
Sep 15 21:18:35.340189 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:35.340122    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g" podStartSLOduration=1.340103886 podStartE2EDuration="1.340103886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:18:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:18:35.339612721 +0000 UTC m=+1251.881775387" watchObservedRunningTime="2026-09-15 21:18:35.340103886 +0000 UTC m=+1251.882266522"
Sep 15 21:18:35.739966 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:35.739928    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:35.842173 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:35.842134    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:36.584748 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:36.584714    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:37.336545 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:37.336507    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:38.093159 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:38.093120    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:39.047981 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:39.047937    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:39.502014 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:39.501977    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:39.755258 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:39.755167    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:39.858345 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:39.858292    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:40.141097 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:40.141062    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:40.371869 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:40.371831    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:43.361056 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:43.361019    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:43.533706 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:43.533664    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:43.641973 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:43.641868    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:44.288911 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:44.288881    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:44.619633 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:44.619598    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:44.620089 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:44.619906    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:44.632744 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:44.632707    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:45.078658 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:45.078558    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:45.385646 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:45.385613    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:18:45.997864 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:45.997827    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:46.781608 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:46.781573    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:47.891082 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:47.891025    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:48.046966 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:48.046930    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:48.166758 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:48.166657    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:48.769076 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:48.769044    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:48.865656 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:48.865618    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:18:48.884367 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:48.884332    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb"]
Sep 15 21:18:48.888687 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:48.888667    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb"
Sep 15 21:18:48.893427 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:48.893406    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-41ffdf-dockercfg-nhs29\""
Sep 15 21:18:48.893783 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:48.893704    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-41ffdf\""
Sep 15 21:18:48.903684 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:48.903659    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb"]
Sep 15 21:18:48.937564 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:48.937520    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpsrg\" (UniqueName: \"kubernetes.io/projected/deb8a238-e92b-435e-964b-fee577c19776-kube-api-access-gpsrg\") pod \"payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb\" (UID: \"deb8a238-e92b-435e-964b-fee577c19776\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb"
Sep 15 21:18:48.937564 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:48.937561    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/deb8a238-e92b-435e-964b-fee577c19776-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb\" (UID: \"deb8a238-e92b-435e-964b-fee577c19776\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb"
Sep 15 21:18:49.039134 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:49.039039    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpsrg\" (UniqueName: \"kubernetes.io/projected/deb8a238-e92b-435e-964b-fee577c19776-kube-api-access-gpsrg\") pod \"payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb\" (UID: \"deb8a238-e92b-435e-964b-fee577c19776\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb"
Sep 15 21:18:49.039323 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:49.039254    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/deb8a238-e92b-435e-964b-fee577c19776-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb\" (UID: \"deb8a238-e92b-435e-964b-fee577c19776\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb"
Sep 15 21:18:49.039969 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:49.039947    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/deb8a238-e92b-435e-964b-fee577c19776-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb\" (UID: \"deb8a238-e92b-435e-964b-fee577c19776\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb"
Sep 15 21:18:49.058629 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:49.058597    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpsrg\" (UniqueName: \"kubernetes.io/projected/deb8a238-e92b-435e-964b-fee577c19776-kube-api-access-gpsrg\") pod \"payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb\" (UID: \"deb8a238-e92b-435e-964b-fee577c19776\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb"
Sep 15 21:18:49.199997 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:49.199962    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb"
Sep 15 21:18:49.474850 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:49.474822    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb"]
Sep 15 21:18:49.476928 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:18:49.476900    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeb8a238_e92b_435e_964b_fee577c19776.slice/crio-41304bec3d48404e18a855bf16acad3095f241f7ce1f9f526f3a58ae559afcb6 WatchSource:0}: Error finding container 41304bec3d48404e18a855bf16acad3095f241f7ce1f9f526f3a58ae559afcb6: Status 404 returned error can't find the container with id 41304bec3d48404e18a855bf16acad3095f241f7ce1f9f526f3a58ae559afcb6
Sep 15 21:18:50.406080 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:50.406043    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb" event={"ID":"deb8a238-e92b-435e-964b-fee577c19776","Type":"ContainerStarted","Data":"345256b9bc545bb0d3ca33431da2355fade26b4ac840f5488ff1f77cd1823a80"}
Sep 15 21:18:50.406524 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:50.406103    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb"
Sep 15 21:18:50.406524 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:50.406126    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb" event={"ID":"deb8a238-e92b-435e-964b-fee577c19776","Type":"ContainerStarted","Data":"41304bec3d48404e18a855bf16acad3095f241f7ce1f9f526f3a58ae559afcb6"}
Sep 15 21:18:50.426340 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:18:50.426288    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb" podStartSLOduration=2.426273049 podStartE2EDuration="2.426273049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:18:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:18:50.424185555 +0000 UTC m=+1266.966348198" watchObservedRunningTime="2026-09-15 21:18:50.426273049 +0000 UTC m=+1266.968435684"
Sep 15 21:19:01.330565 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:01.330527    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:01.413872 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:01.413844    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb"
Sep 15 21:19:02.330145 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:02.330028    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:02.999621 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:02.999582    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:03.290698 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:03.290602    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:03.925860 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:03.925821    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:04.769977 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:04.769934    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:05.593272 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:05.593233    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:06.308998 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:06.308952    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:07.213204 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:07.213169    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:07.959140 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:07.959103    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:08.954535 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:08.954476    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:09.702559 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:09.702512    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:11.105868 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:11.105831    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:12.645265 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:12.645233    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:16.731595 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:16.731558    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"]
Sep 15 21:19:16.732102 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:16.731901    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8" podUID="4b4b628b-5962-423b-bfeb-5c63ec34caeb" containerName="maas-api" containerID="cri-o://5fffc74e43fabf574224c7d3dfa308d8fb2117713fa4fcf39c88c7c6a3aa1573" gracePeriod=30
Sep 15 21:19:16.753961 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:16.753926    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:16.853185 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:16.853153    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:16.987027 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:16.986963    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"
Sep 15 21:19:16.992034 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:16.992000    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:17.002527 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.002474    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b4b628b-5962-423b-bfeb-5c63ec34caeb-maas-api-tls\") pod \"4b4b628b-5962-423b-bfeb-5c63ec34caeb\" (UID: \"4b4b628b-5962-423b-bfeb-5c63ec34caeb\") "
Sep 15 21:19:17.002662 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.002550    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b4b628b-5962-423b-bfeb-5c63ec34caeb-metrics-certs\") pod \"4b4b628b-5962-423b-bfeb-5c63ec34caeb\" (UID: \"4b4b628b-5962-423b-bfeb-5c63ec34caeb\") "
Sep 15 21:19:17.002662 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.002655    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7vrg\" (UniqueName: \"kubernetes.io/projected/4b4b628b-5962-423b-bfeb-5c63ec34caeb-kube-api-access-n7vrg\") pod \"4b4b628b-5962-423b-bfeb-5c63ec34caeb\" (UID: \"4b4b628b-5962-423b-bfeb-5c63ec34caeb\") "
Sep 15 21:19:17.002748 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.002694    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b4b628b-5962-423b-bfeb-5c63ec34caeb-openshift-service-ca\") pod \"4b4b628b-5962-423b-bfeb-5c63ec34caeb\" (UID: \"4b4b628b-5962-423b-bfeb-5c63ec34caeb\") "
Sep 15 21:19:17.003115 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.003082    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b4b628b-5962-423b-bfeb-5c63ec34caeb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4b4b628b-5962-423b-bfeb-5c63ec34caeb" (UID: "4b4b628b-5962-423b-bfeb-5c63ec34caeb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:19:17.004898 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.004870    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b4b628b-5962-423b-bfeb-5c63ec34caeb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4b4b628b-5962-423b-bfeb-5c63ec34caeb" (UID: "4b4b628b-5962-423b-bfeb-5c63ec34caeb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:19:17.005335 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.005313    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b4b628b-5962-423b-bfeb-5c63ec34caeb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4b4b628b-5962-423b-bfeb-5c63ec34caeb" (UID: "4b4b628b-5962-423b-bfeb-5c63ec34caeb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:19:17.005410 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.005318    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b4b628b-5962-423b-bfeb-5c63ec34caeb-kube-api-access-n7vrg" (OuterVolumeSpecName: "kube-api-access-n7vrg") pod "4b4b628b-5962-423b-bfeb-5c63ec34caeb" (UID: "4b4b628b-5962-423b-bfeb-5c63ec34caeb"). InnerVolumeSpecName "kube-api-access-n7vrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:19:17.084569 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.084532    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:17.103789 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.103755    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7vrg\" (UniqueName: \"kubernetes.io/projected/4b4b628b-5962-423b-bfeb-5c63ec34caeb-kube-api-access-n7vrg\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:19:17.103789 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.103783    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b4b628b-5962-423b-bfeb-5c63ec34caeb-openshift-service-ca\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:19:17.103789 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.103793    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b4b628b-5962-423b-bfeb-5c63ec34caeb-maas-api-tls\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:19:17.104022 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.103801    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b4b628b-5962-423b-bfeb-5c63ec34caeb-metrics-certs\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:19:17.546896 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.546853    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:17.550501 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.550463    2564 generic.go:356] "Generic (PLEG): container finished" podID="4b4b628b-5962-423b-bfeb-5c63ec34caeb" containerID="5fffc74e43fabf574224c7d3dfa308d8fb2117713fa4fcf39c88c7c6a3aa1573" exitCode=0
Sep 15 21:19:17.550645 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.550524    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"
Sep 15 21:19:17.550645 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.550548    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8" event={"ID":"4b4b628b-5962-423b-bfeb-5c63ec34caeb","Type":"ContainerDied","Data":"5fffc74e43fabf574224c7d3dfa308d8fb2117713fa4fcf39c88c7c6a3aa1573"}
Sep 15 21:19:17.550645 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.550606    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8" event={"ID":"4b4b628b-5962-423b-bfeb-5c63ec34caeb","Type":"ContainerDied","Data":"69df5006870575ea19142d4633638fe1d32fe351d92ca0b84af831a4973a6473"}
Sep 15 21:19:17.550645 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.550628    2564 scope.go:117] "RemoveContainer" containerID="5fffc74e43fabf574224c7d3dfa308d8fb2117713fa4fcf39c88c7c6a3aa1573"
Sep 15 21:19:17.560397 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.560379    2564 scope.go:117] "RemoveContainer" containerID="5fffc74e43fabf574224c7d3dfa308d8fb2117713fa4fcf39c88c7c6a3aa1573"
Sep 15 21:19:17.560753 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:19:17.560733    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fffc74e43fabf574224c7d3dfa308d8fb2117713fa4fcf39c88c7c6a3aa1573\": container with ID starting with 5fffc74e43fabf574224c7d3dfa308d8fb2117713fa4fcf39c88c7c6a3aa1573 not found: ID does not exist" containerID="5fffc74e43fabf574224c7d3dfa308d8fb2117713fa4fcf39c88c7c6a3aa1573"
Sep 15 21:19:17.560839 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.560767    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fffc74e43fabf574224c7d3dfa308d8fb2117713fa4fcf39c88c7c6a3aa1573"} err="failed to get container status \"5fffc74e43fabf574224c7d3dfa308d8fb2117713fa4fcf39c88c7c6a3aa1573\": rpc error: code = NotFound desc = could not find container \"5fffc74e43fabf574224c7d3dfa308d8fb2117713fa4fcf39c88c7c6a3aa1573\": container with ID starting with 5fffc74e43fabf574224c7d3dfa308d8fb2117713fa4fcf39c88c7c6a3aa1573 not found: ID does not exist"
Sep 15 21:19:17.572897 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.572867    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"]
Sep 15 21:19:17.576276 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.576253    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-56c6b012-58978d5c8f-ftrx8"]
Sep 15 21:19:17.973710 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:17.973664    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b4b628b-5962-423b-bfeb-5c63ec34caeb" path="/var/lib/kubelet/pods/4b4b628b-5962-423b-bfeb-5c63ec34caeb/volumes"
Sep 15 21:19:18.198163 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:18.198129    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:19.130680 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:19.130641    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:19.846239 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:19.846204    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:20.097320 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:20.097287    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:20.681163 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:20.681133    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:21.463945 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:21.463903    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:25.100714 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:25.100679    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:49.118622 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:49.118578    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:19:49.219381 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:19:49.219343    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:01.933022 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:01.932983    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:02.397331 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:02.397291    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:04.783794 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:04.783718    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:04.842706 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:04.842664    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:08.182713 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:08.182678    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:13.791732 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:13.791697    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:13.856566 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:13.856512    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:14.051229 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:14.051142    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:15.275896 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:15.275859    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:15.374578 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:15.374536    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:15.471559 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:15.471523    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:16.013690 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:16.013653    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:16.107662 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:16.107620    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:16.211025 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:16.210994    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:17.421619 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:17.421567    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:19.814067 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:19.814027    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:23.209216 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:23.209183    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:23.311084 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:23.310989    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:23.861977 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:23.861927    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:29.929758 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:29.929716    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:30.036318 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:30.036277    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:30.165887 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:30.165854    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:37.094917 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.094882    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"]
Sep 15 21:20:37.095380 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.095364    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b4b628b-5962-423b-bfeb-5c63ec34caeb" containerName="maas-api"
Sep 15 21:20:37.095380 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.095382    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b4b628b-5962-423b-bfeb-5c63ec34caeb" containerName="maas-api"
Sep 15 21:20:37.095478 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.095463    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b4b628b-5962-423b-bfeb-5c63ec34caeb" containerName="maas-api"
Sep 15 21:20:37.098862 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.098842    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:37.101624 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.101598    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-544890\"/\"embedding-w0-544890-kserve-self-signed-certs\""
Sep 15 21:20:37.108438 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.108409    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"]
Sep 15 21:20:37.120675 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.120640    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:37.199821 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.199784    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2473cc6b-0bf7-42b7-8689-ccc6bd736912-tls-certs\") pod \"embedding-w0-544890-kserve-5f8c78867b-2rvt9\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") " pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:37.200021 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.199868    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-dshm\") pod \"embedding-w0-544890-kserve-5f8c78867b-2rvt9\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") " pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:37.200021 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.199888    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-tmp-dir\") pod \"embedding-w0-544890-kserve-5f8c78867b-2rvt9\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") " pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:37.200021 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.199915    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-model-cache\") pod \"embedding-w0-544890-kserve-5f8c78867b-2rvt9\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") " pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:37.200021 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.199994    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtjmx\" (UniqueName: \"kubernetes.io/projected/2473cc6b-0bf7-42b7-8689-ccc6bd736912-kube-api-access-wtjmx\") pod \"embedding-w0-544890-kserve-5f8c78867b-2rvt9\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") " pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:37.200021 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.200018    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-home\") pod \"embedding-w0-544890-kserve-5f8c78867b-2rvt9\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") " pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:37.300833 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.300794    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2473cc6b-0bf7-42b7-8689-ccc6bd736912-tls-certs\") pod \"embedding-w0-544890-kserve-5f8c78867b-2rvt9\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") " pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:37.301038 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.300902    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-dshm\") pod \"embedding-w0-544890-kserve-5f8c78867b-2rvt9\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") " pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:37.301038 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.300923    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-tmp-dir\") pod \"embedding-w0-544890-kserve-5f8c78867b-2rvt9\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") " pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:37.301038 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.300953    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-model-cache\") pod \"embedding-w0-544890-kserve-5f8c78867b-2rvt9\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") " pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:37.301038 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.301008    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtjmx\" (UniqueName: \"kubernetes.io/projected/2473cc6b-0bf7-42b7-8689-ccc6bd736912-kube-api-access-wtjmx\") pod \"embedding-w0-544890-kserve-5f8c78867b-2rvt9\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") " pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:37.301255 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.301049    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-home\") pod \"embedding-w0-544890-kserve-5f8c78867b-2rvt9\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") " pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:37.301479 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.301445    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-tmp-dir\") pod \"embedding-w0-544890-kserve-5f8c78867b-2rvt9\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") " pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:37.301479 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.301461    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-model-cache\") pod \"embedding-w0-544890-kserve-5f8c78867b-2rvt9\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") " pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:37.301668 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.301532    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-home\") pod \"embedding-w0-544890-kserve-5f8c78867b-2rvt9\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") " pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:37.303309 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.303286    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-dshm\") pod \"embedding-w0-544890-kserve-5f8c78867b-2rvt9\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") " pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:37.303470 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.303454    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2473cc6b-0bf7-42b7-8689-ccc6bd736912-tls-certs\") pod \"embedding-w0-544890-kserve-5f8c78867b-2rvt9\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") " pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:37.310089 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.310065    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtjmx\" (UniqueName: \"kubernetes.io/projected/2473cc6b-0bf7-42b7-8689-ccc6bd736912-kube-api-access-wtjmx\") pod \"embedding-w0-544890-kserve-5f8c78867b-2rvt9\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") " pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:37.410244 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.410207    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:37.684363 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.684335    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"]
Sep 15 21:20:37.686207 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:20:37.686170    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2473cc6b_0bf7_42b7_8689_ccc6bd736912.slice/crio-ff2c4059286f37c766f58eef63eb842cb81d4649aba42d9b65833bedc2e48adf WatchSource:0}: Error finding container ff2c4059286f37c766f58eef63eb842cb81d4649aba42d9b65833bedc2e48adf: Status 404 returned error can't find the container with id ff2c4059286f37c766f58eef63eb842cb81d4649aba42d9b65833bedc2e48adf
Sep 15 21:20:37.977147 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.977051    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9" event={"ID":"2473cc6b-0bf7-42b7-8689-ccc6bd736912","Type":"ContainerStarted","Data":"2117e6189bc9996ac8ae015aeeb6192060c12f97270f8af271137d647b8d6491"}
Sep 15 21:20:37.977147 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.977093    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9" event={"ID":"2473cc6b-0bf7-42b7-8689-ccc6bd736912","Type":"ContainerStarted","Data":"ff2c4059286f37c766f58eef63eb842cb81d4649aba42d9b65833bedc2e48adf"}
Sep 15 21:20:37.994794 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:37.994743    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9" podStartSLOduration=0.994726397 podStartE2EDuration="994.726397ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:20:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:20:37.99458825 +0000 UTC m=+1374.536750904" watchObservedRunningTime="2026-09-15 21:20:37.994726397 +0000 UTC m=+1374.536889032"
Sep 15 21:20:40.618536 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:40.618482    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:40.891560 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:40.891463    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:20:47.411069 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:47.411021    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:47.411069 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:47.411080    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:47.423754 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:47.423719    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:48.041367 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:48.041333    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:20:54.863723 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:20:54.863686    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:01.767858 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:01.767815    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:07.682030 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:07.681988    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:08.058601 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:08.058478    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:08.161706 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:08.161667    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:08.317983 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:08.317900    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:08.427804 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:08.427767    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:11.290574 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:11.290530    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:11.689560 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:11.689525    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:12.048357 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:12.048254    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:12.371262 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:12.371218    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:13.163393 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:13.163355    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:14.079929 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:14.079889    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:14.805839 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:14.805804    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:15.517241 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:15.517200    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:15.576421 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:15.576386    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:15.848275 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:15.848240    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:16.171869 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:16.171766    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:16.265185 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:16.265149    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:17.562629 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:17.562593    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:17.663676 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:17.663645    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:18.458240 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:18.458208    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:19.263326 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:19.263288    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:19.734801 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:19.734758    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:20.460611 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:20.460559    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:21.400909 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:21.400877    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:22.099335 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:22.099297    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:23.008063 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:23.008019    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:23.573559 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:23.573524    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:24.257184 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:24.257150    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"]
Sep 15 21:21:24.263060 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:24.263027    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"]
Sep 15 21:21:24.275585 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:24.275550    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"]
Sep 15 21:21:24.291136 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:24.291018    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:24.297154 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:24.295147    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"]
Sep 15 21:21:24.369730 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:24.369697    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:25.184908 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:25.184866    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:28.357819 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:28.357776    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:28.436978 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:28.436944    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:34.947521 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:34.947412    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"]
Sep 15 21:21:34.947990 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:34.947769    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds" podUID="b27dd28b-9a41-4a3e-a821-12ad8fb7a413" containerName="maas-api" containerID="cri-o://ed5aef2f3cdb674d2237970a80c3c93470cca7d76c5669e82f45d11440c36dee" gracePeriod=30
Sep 15 21:21:34.976543 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:34.976513    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:35.000900 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.000860    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v"]
Sep 15 21:21:35.001153 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.001118    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v" podUID="f5a525f4-a59e-4444-a998-453225e4f1e4" containerName="payload-processing" containerID="cri-o://695b96c29d40700fc14143ed0ba9772dfd019c4f876c8611f0d33c1f0bcb4a5d" gracePeriod=30
Sep 15 21:21:35.009095 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.009062    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2"]
Sep 15 21:21:35.009372 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.009318    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2" podUID="c8779115-0e1b-4b97-81c6-501c16ce4412" containerName="payload-pre-processing" containerID="cri-o://2bb99b311c4c698ef5e8a5c3ead60e465d0e877f9448ffa3c9143580c50f005d" gracePeriod=30
Sep 15 21:21:35.043309 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.043276    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:35.145324 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.145292    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:35.246212 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.246085    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:35.282792 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.282762    2564 generic.go:356] "Generic (PLEG): container finished" podID="c8779115-0e1b-4b97-81c6-501c16ce4412" containerID="2bb99b311c4c698ef5e8a5c3ead60e465d0e877f9448ffa3c9143580c50f005d" exitCode=0
Sep 15 21:21:35.282944 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.282847    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2" event={"ID":"c8779115-0e1b-4b97-81c6-501c16ce4412","Type":"ContainerDied","Data":"2bb99b311c4c698ef5e8a5c3ead60e465d0e877f9448ffa3c9143580c50f005d"}
Sep 15 21:21:35.284587 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.284559    2564 generic.go:356] "Generic (PLEG): container finished" podID="f5a525f4-a59e-4444-a998-453225e4f1e4" containerID="695b96c29d40700fc14143ed0ba9772dfd019c4f876c8611f0d33c1f0bcb4a5d" exitCode=0
Sep 15 21:21:35.284709 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.284692    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v" event={"ID":"f5a525f4-a59e-4444-a998-453225e4f1e4","Type":"ContainerDied","Data":"695b96c29d40700fc14143ed0ba9772dfd019c4f876c8611f0d33c1f0bcb4a5d"}
Sep 15 21:21:35.286567 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.286371    2564 generic.go:356] "Generic (PLEG): container finished" podID="b27dd28b-9a41-4a3e-a821-12ad8fb7a413" containerID="ed5aef2f3cdb674d2237970a80c3c93470cca7d76c5669e82f45d11440c36dee" exitCode=0
Sep 15 21:21:35.286567 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.286399    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds" event={"ID":"b27dd28b-9a41-4a3e-a821-12ad8fb7a413","Type":"ContainerDied","Data":"ed5aef2f3cdb674d2237970a80c3c93470cca7d76c5669e82f45d11440c36dee"}
Sep 15 21:21:35.286567 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.286464    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds" event={"ID":"b27dd28b-9a41-4a3e-a821-12ad8fb7a413","Type":"ContainerDied","Data":"33b07a932ebfd18cbc1f838933af4190ffcaba7efd07adba20dddd2a3d786bfc"}
Sep 15 21:21:35.286567 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.286513    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="33b07a932ebfd18cbc1f838933af4190ffcaba7efd07adba20dddd2a3d786bfc"
Sep 15 21:21:35.302726 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.302396    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"
Sep 15 21:21:35.308309 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.308279    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v"
Sep 15 21:21:35.324804 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.324770    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2"
Sep 15 21:21:35.395438 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.395407    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5a525f4-a59e-4444-a998-453225e4f1e4-plugins-config-volume\") pod \"f5a525f4-a59e-4444-a998-453225e4f1e4\" (UID: \"f5a525f4-a59e-4444-a998-453225e4f1e4\") "
Sep 15 21:21:35.395679 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.395452    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4799p\" (UniqueName: \"kubernetes.io/projected/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-kube-api-access-4799p\") pod \"b27dd28b-9a41-4a3e-a821-12ad8fb7a413\" (UID: \"b27dd28b-9a41-4a3e-a821-12ad8fb7a413\") "
Sep 15 21:21:35.395679 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.395475    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-openshift-service-ca\") pod \"b27dd28b-9a41-4a3e-a821-12ad8fb7a413\" (UID: \"b27dd28b-9a41-4a3e-a821-12ad8fb7a413\") "
Sep 15 21:21:35.395679 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.395525    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4shj\" (UniqueName: \"kubernetes.io/projected/f5a525f4-a59e-4444-a998-453225e4f1e4-kube-api-access-v4shj\") pod \"f5a525f4-a59e-4444-a998-453225e4f1e4\" (UID: \"f5a525f4-a59e-4444-a998-453225e4f1e4\") "
Sep 15 21:21:35.395679 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.395548    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-metrics-certs\") pod \"b27dd28b-9a41-4a3e-a821-12ad8fb7a413\" (UID: \"b27dd28b-9a41-4a3e-a821-12ad8fb7a413\") "
Sep 15 21:21:35.395679 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.395591    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-maas-api-tls\") pod \"b27dd28b-9a41-4a3e-a821-12ad8fb7a413\" (UID: \"b27dd28b-9a41-4a3e-a821-12ad8fb7a413\") "
Sep 15 21:21:35.395962 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.395873    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5a525f4-a59e-4444-a998-453225e4f1e4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f5a525f4-a59e-4444-a998-453225e4f1e4" (UID: "f5a525f4-a59e-4444-a998-453225e4f1e4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:21:35.396206 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.396177    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b27dd28b-9a41-4a3e-a821-12ad8fb7a413" (UID: "b27dd28b-9a41-4a3e-a821-12ad8fb7a413"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:21:35.396545 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.396523    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5a525f4-a59e-4444-a998-453225e4f1e4-plugins-config-volume\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:35.396633 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.396549    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-openshift-service-ca\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:35.398050 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.398026    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b27dd28b-9a41-4a3e-a821-12ad8fb7a413" (UID: "b27dd28b-9a41-4a3e-a821-12ad8fb7a413"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:21:35.398050 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.398035    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5a525f4-a59e-4444-a998-453225e4f1e4-kube-api-access-v4shj" (OuterVolumeSpecName: "kube-api-access-v4shj") pod "f5a525f4-a59e-4444-a998-453225e4f1e4" (UID: "f5a525f4-a59e-4444-a998-453225e4f1e4"). InnerVolumeSpecName "kube-api-access-v4shj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:21:35.398304 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.398284    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b27dd28b-9a41-4a3e-a821-12ad8fb7a413" (UID: "b27dd28b-9a41-4a3e-a821-12ad8fb7a413"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:21:35.398362 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.398324    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-kube-api-access-4799p" (OuterVolumeSpecName: "kube-api-access-4799p") pod "b27dd28b-9a41-4a3e-a821-12ad8fb7a413" (UID: "b27dd28b-9a41-4a3e-a821-12ad8fb7a413"). InnerVolumeSpecName "kube-api-access-4799p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:21:35.497385 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.497296    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkqgk\" (UniqueName: \"kubernetes.io/projected/c8779115-0e1b-4b97-81c6-501c16ce4412-kube-api-access-qkqgk\") pod \"c8779115-0e1b-4b97-81c6-501c16ce4412\" (UID: \"c8779115-0e1b-4b97-81c6-501c16ce4412\") "
Sep 15 21:21:35.497385 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.497332    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8779115-0e1b-4b97-81c6-501c16ce4412-plugins-config-volume\") pod \"c8779115-0e1b-4b97-81c6-501c16ce4412\" (UID: \"c8779115-0e1b-4b97-81c6-501c16ce4412\") "
Sep 15 21:21:35.497701 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.497683    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4799p\" (UniqueName: \"kubernetes.io/projected/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-kube-api-access-4799p\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:35.497701 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.497701    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4shj\" (UniqueName: \"kubernetes.io/projected/f5a525f4-a59e-4444-a998-453225e4f1e4-kube-api-access-v4shj\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:35.497851 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.497710    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-metrics-certs\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:35.497851 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.497718    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b27dd28b-9a41-4a3e-a821-12ad8fb7a413-maas-api-tls\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:35.497851 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.497819    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8779115-0e1b-4b97-81c6-501c16ce4412-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c8779115-0e1b-4b97-81c6-501c16ce4412" (UID: "c8779115-0e1b-4b97-81c6-501c16ce4412"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:21:35.499605 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.499579    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8779115-0e1b-4b97-81c6-501c16ce4412-kube-api-access-qkqgk" (OuterVolumeSpecName: "kube-api-access-qkqgk") pod "c8779115-0e1b-4b97-81c6-501c16ce4412" (UID: "c8779115-0e1b-4b97-81c6-501c16ce4412"). InnerVolumeSpecName "kube-api-access-qkqgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:21:35.540876 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.540829    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:35.599099 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.599062    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkqgk\" (UniqueName: \"kubernetes.io/projected/c8779115-0e1b-4b97-81c6-501c16ce4412-kube-api-access-qkqgk\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:35.599099 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.599091    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8779115-0e1b-4b97-81c6-501c16ce4412-plugins-config-volume\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:35.641623 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:35.641586    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:36.293529 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:36.293424    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2"
Sep 15 21:21:36.293529 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:36.293435    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2" event={"ID":"c8779115-0e1b-4b97-81c6-501c16ce4412","Type":"ContainerDied","Data":"4809072c086a185d12c8cb6209657ef7a5896987ec84cd800d16d7215e760fa9"}
Sep 15 21:21:36.294049 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:36.293548    2564 scope.go:117] "RemoveContainer" containerID="2bb99b311c4c698ef5e8a5c3ead60e465d0e877f9448ffa3c9143580c50f005d"
Sep 15 21:21:36.295586 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:36.295565    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v"
Sep 15 21:21:36.295882 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:36.295595    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v" event={"ID":"f5a525f4-a59e-4444-a998-453225e4f1e4","Type":"ContainerDied","Data":"5a8f3cdef2a34833e4e5043de7f1d8c31c93f41be8e410d68c2328434bd8f2be"}
Sep 15 21:21:36.296029 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:36.295654    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"
Sep 15 21:21:36.304626 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:36.304606    2564 scope.go:117] "RemoveContainer" containerID="695b96c29d40700fc14143ed0ba9772dfd019c4f876c8611f0d33c1f0bcb4a5d"
Sep 15 21:21:36.311446 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:36.311418    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2"]
Sep 15 21:21:36.315752 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:36.315731    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-544890-79d99db4fd-mnlf2"]
Sep 15 21:21:36.332774 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:36.332747    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"]
Sep 15 21:21:36.337134 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:36.337101    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-544890-7756874b8c-2vqds"]
Sep 15 21:21:36.347999 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:36.347975    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v"]
Sep 15 21:21:36.351390 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:36.351360    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-544890-5cf9fd9747-w6n6v"]
Sep 15 21:21:36.489684 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:36.489639    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:37.309924 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:37.309889    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:37.973807 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:37.973772    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b27dd28b-9a41-4a3e-a821-12ad8fb7a413" path="/var/lib/kubelet/pods/b27dd28b-9a41-4a3e-a821-12ad8fb7a413/volumes"
Sep 15 21:21:37.974137 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:37.974124    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8779115-0e1b-4b97-81c6-501c16ce4412" path="/var/lib/kubelet/pods/c8779115-0e1b-4b97-81c6-501c16ce4412/volumes"
Sep 15 21:21:37.974436 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:37.974425    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5a525f4-a59e-4444-a998-453225e4f1e4" path="/var/lib/kubelet/pods/f5a525f4-a59e-4444-a998-453225e4f1e4/volumes"
Sep 15 21:21:38.026669 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:38.026628    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:38.989776 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:38.989733    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:39.278076 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.277978    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g" podUID="784be271-ba64-4757-9c89-cc238f0e3f22" containerName="main" containerID="cri-o://f3e46b0c9a5346172d5dacba28fe9f7a08c98b8dc617acf58eba15441cea7d29" gracePeriod=45
Sep 15 21:21:39.283737 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.283693    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg" podUID="37bfe67c-8ca3-4ea1-916f-8012b9ec3195" containerName="main" containerID="cri-o://f8c77b83856f1439774c439b5c19ea61aff299edf4c033dd39e15fd21f765d15" gracePeriod=45
Sep 15 21:21:39.298270 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.298229    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9" podUID="2473cc6b-0bf7-42b7-8689-ccc6bd736912" containerName="main" containerID="cri-o://2117e6189bc9996ac8ae015aeeb6192060c12f97270f8af271137d647b8d6491" gracePeriod=45
Sep 15 21:21:39.322335 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.322288    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn" podUID="6702c4b7-5609-496b-808f-9a227938f6ad" containerName="main" containerID="cri-o://b8d656a0639ea4b149472189a6d540330bc8b657c0de90590007285e62c8eafe" gracePeriod=45
Sep 15 21:21:39.627526 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.627479    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:21:39.641800 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.641569    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:21:39.653277 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.653243    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:21:39.657675 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.657645    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:21:39.741361 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.741330    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29q5t\" (UniqueName: \"kubernetes.io/projected/6702c4b7-5609-496b-808f-9a227938f6ad-kube-api-access-29q5t\") pod \"6702c4b7-5609-496b-808f-9a227938f6ad\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") "
Sep 15 21:21:39.741568 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.741402    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzjsw\" (UniqueName: \"kubernetes.io/projected/784be271-ba64-4757-9c89-cc238f0e3f22-kube-api-access-qzjsw\") pod \"784be271-ba64-4757-9c89-cc238f0e3f22\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") "
Sep 15 21:21:39.741648 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.741570    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-model-cache\") pod \"784be271-ba64-4757-9c89-cc238f0e3f22\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") "
Sep 15 21:21:39.741648 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.741629    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-tmp-dir\") pod \"6702c4b7-5609-496b-808f-9a227938f6ad\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") "
Sep 15 21:21:39.741760 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.741669    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-tmp-dir\") pod \"784be271-ba64-4757-9c89-cc238f0e3f22\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") "
Sep 15 21:21:39.741760 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.741702    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-dshm\") pod \"6702c4b7-5609-496b-808f-9a227938f6ad\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") "
Sep 15 21:21:39.741862 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.741777    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-dshm\") pod \"784be271-ba64-4757-9c89-cc238f0e3f22\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") "
Sep 15 21:21:39.741862 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.741807    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-model-cache\") pod \"6702c4b7-5609-496b-808f-9a227938f6ad\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") "
Sep 15 21:21:39.741862 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.741822    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6702c4b7-5609-496b-808f-9a227938f6ad" (UID: "6702c4b7-5609-496b-808f-9a227938f6ad"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:21:39.741862 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.741843    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/784be271-ba64-4757-9c89-cc238f0e3f22-tls-certs\") pod \"784be271-ba64-4757-9c89-cc238f0e3f22\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") "
Sep 15 21:21:39.741862 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.741845    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-model-cache" (OuterVolumeSpecName: "model-cache") pod "784be271-ba64-4757-9c89-cc238f0e3f22" (UID: "784be271-ba64-4757-9c89-cc238f0e3f22"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:21:39.742109 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.741875    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-home\") pod \"784be271-ba64-4757-9c89-cc238f0e3f22\" (UID: \"784be271-ba64-4757-9c89-cc238f0e3f22\") "
Sep 15 21:21:39.742109 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.741936    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-home\") pod \"6702c4b7-5609-496b-808f-9a227938f6ad\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") "
Sep 15 21:21:39.742109 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.741963    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6702c4b7-5609-496b-808f-9a227938f6ad-tls-certs\") pod \"6702c4b7-5609-496b-808f-9a227938f6ad\" (UID: \"6702c4b7-5609-496b-808f-9a227938f6ad\") "
Sep 15 21:21:39.742109 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.741972    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "784be271-ba64-4757-9c89-cc238f0e3f22" (UID: "784be271-ba64-4757-9c89-cc238f0e3f22"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:21:39.742109 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.742032    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-model-cache" (OuterVolumeSpecName: "model-cache") pod "6702c4b7-5609-496b-808f-9a227938f6ad" (UID: "6702c4b7-5609-496b-808f-9a227938f6ad"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:21:39.742322 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.742202    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-home" (OuterVolumeSpecName: "home") pod "784be271-ba64-4757-9c89-cc238f0e3f22" (UID: "784be271-ba64-4757-9c89-cc238f0e3f22"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:21:39.743882 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.743018    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-home" (OuterVolumeSpecName: "home") pod "6702c4b7-5609-496b-808f-9a227938f6ad" (UID: "6702c4b7-5609-496b-808f-9a227938f6ad"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:21:39.743882 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.743202    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-model-cache\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.743882 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.743218    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-home\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.743882 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.743231    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-home\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.743882 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.743248    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-model-cache\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.743882 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.743260    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-tmp-dir\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.743882 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.743270    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-tmp-dir\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.747345 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.744840    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-dshm" (OuterVolumeSpecName: "dshm") pod "784be271-ba64-4757-9c89-cc238f0e3f22" (UID: "784be271-ba64-4757-9c89-cc238f0e3f22"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:21:39.747345 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.744947    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6702c4b7-5609-496b-808f-9a227938f6ad-kube-api-access-29q5t" (OuterVolumeSpecName: "kube-api-access-29q5t") pod "6702c4b7-5609-496b-808f-9a227938f6ad" (UID: "6702c4b7-5609-496b-808f-9a227938f6ad"). InnerVolumeSpecName "kube-api-access-29q5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:21:39.747345 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.746386    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6702c4b7-5609-496b-808f-9a227938f6ad-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6702c4b7-5609-496b-808f-9a227938f6ad" (UID: "6702c4b7-5609-496b-808f-9a227938f6ad"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:21:39.753528 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.753476    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/784be271-ba64-4757-9c89-cc238f0e3f22-kube-api-access-qzjsw" (OuterVolumeSpecName: "kube-api-access-qzjsw") pod "784be271-ba64-4757-9c89-cc238f0e3f22" (UID: "784be271-ba64-4757-9c89-cc238f0e3f22"). InnerVolumeSpecName "kube-api-access-qzjsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:21:39.753901 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.753871    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/784be271-ba64-4757-9c89-cc238f0e3f22-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "784be271-ba64-4757-9c89-cc238f0e3f22" (UID: "784be271-ba64-4757-9c89-cc238f0e3f22"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:21:39.754104 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.754071    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-dshm" (OuterVolumeSpecName: "dshm") pod "6702c4b7-5609-496b-808f-9a227938f6ad" (UID: "6702c4b7-5609-496b-808f-9a227938f6ad"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:21:39.844551 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.844416    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64462\" (UniqueName: \"kubernetes.io/projected/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-kube-api-access-64462\") pod \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") "
Sep 15 21:21:39.844551 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.844458    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-dshm\") pod \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") "
Sep 15 21:21:39.844551 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.844474    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-home\") pod \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") "
Sep 15 21:21:39.844551 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.844517    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-tmp-dir\") pod \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") "
Sep 15 21:21:39.844551 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.844550    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2473cc6b-0bf7-42b7-8689-ccc6bd736912-tls-certs\") pod \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") "
Sep 15 21:21:39.844947 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.844593    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-model-cache\") pod \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") "
Sep 15 21:21:39.844947 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.844627    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-tmp-dir\") pod \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") "
Sep 15 21:21:39.844947 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.844654    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-home\") pod \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") "
Sep 15 21:21:39.844947 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.844690    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-model-cache\") pod \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") "
Sep 15 21:21:39.844947 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.844744    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-tls-certs\") pod \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\" (UID: \"37bfe67c-8ca3-4ea1-916f-8012b9ec3195\") "
Sep 15 21:21:39.844947 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.844777    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-dshm\") pod \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") "
Sep 15 21:21:39.844947 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.844819    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-home" (OuterVolumeSpecName: "home") pod "2473cc6b-0bf7-42b7-8689-ccc6bd736912" (UID: "2473cc6b-0bf7-42b7-8689-ccc6bd736912"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:21:39.844947 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.844833    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtjmx\" (UniqueName: \"kubernetes.io/projected/2473cc6b-0bf7-42b7-8689-ccc6bd736912-kube-api-access-wtjmx\") pod \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\" (UID: \"2473cc6b-0bf7-42b7-8689-ccc6bd736912\") "
Sep 15 21:21:39.844947 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.844868    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "37bfe67c-8ca3-4ea1-916f-8012b9ec3195" (UID: "37bfe67c-8ca3-4ea1-916f-8012b9ec3195"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:21:39.844947 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.844862    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2473cc6b-0bf7-42b7-8689-ccc6bd736912" (UID: "2473cc6b-0bf7-42b7-8689-ccc6bd736912"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:21:39.845647 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.844997    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-model-cache" (OuterVolumeSpecName: "model-cache") pod "37bfe67c-8ca3-4ea1-916f-8012b9ec3195" (UID: "37bfe67c-8ca3-4ea1-916f-8012b9ec3195"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:21:39.845647 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.845169    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-home\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.845647 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.845191    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-tmp-dir\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.845647 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.845206    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6702c4b7-5609-496b-808f-9a227938f6ad-tls-certs\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.845647 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.845221    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29q5t\" (UniqueName: \"kubernetes.io/projected/6702c4b7-5609-496b-808f-9a227938f6ad-kube-api-access-29q5t\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.845647 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.845235    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-tmp-dir\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.845647 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.845247    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-model-cache\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.845647 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.845259    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzjsw\" (UniqueName: \"kubernetes.io/projected/784be271-ba64-4757-9c89-cc238f0e3f22-kube-api-access-qzjsw\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.845647 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.845275    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6702c4b7-5609-496b-808f-9a227938f6ad-dshm\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.845647 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.845318    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/784be271-ba64-4757-9c89-cc238f0e3f22-dshm\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.845647 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.845330    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/784be271-ba64-4757-9c89-cc238f0e3f22-tls-certs\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.845647 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.845378    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-model-cache" (OuterVolumeSpecName: "model-cache") pod "2473cc6b-0bf7-42b7-8689-ccc6bd736912" (UID: "2473cc6b-0bf7-42b7-8689-ccc6bd736912"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:21:39.845647 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.845538    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-home" (OuterVolumeSpecName: "home") pod "37bfe67c-8ca3-4ea1-916f-8012b9ec3195" (UID: "37bfe67c-8ca3-4ea1-916f-8012b9ec3195"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:21:39.847310 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.847283    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-dshm" (OuterVolumeSpecName: "dshm") pod "37bfe67c-8ca3-4ea1-916f-8012b9ec3195" (UID: "37bfe67c-8ca3-4ea1-916f-8012b9ec3195"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:21:39.847310 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.847297    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2473cc6b-0bf7-42b7-8689-ccc6bd736912-kube-api-access-wtjmx" (OuterVolumeSpecName: "kube-api-access-wtjmx") pod "2473cc6b-0bf7-42b7-8689-ccc6bd736912" (UID: "2473cc6b-0bf7-42b7-8689-ccc6bd736912"). InnerVolumeSpecName "kube-api-access-wtjmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:21:39.847528 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.847462    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2473cc6b-0bf7-42b7-8689-ccc6bd736912-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2473cc6b-0bf7-42b7-8689-ccc6bd736912" (UID: "2473cc6b-0bf7-42b7-8689-ccc6bd736912"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:21:39.847592 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.847569    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-kube-api-access-64462" (OuterVolumeSpecName: "kube-api-access-64462") pod "37bfe67c-8ca3-4ea1-916f-8012b9ec3195" (UID: "37bfe67c-8ca3-4ea1-916f-8012b9ec3195"). InnerVolumeSpecName "kube-api-access-64462". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:21:39.847699 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.847683    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-dshm" (OuterVolumeSpecName: "dshm") pod "2473cc6b-0bf7-42b7-8689-ccc6bd736912" (UID: "2473cc6b-0bf7-42b7-8689-ccc6bd736912"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:21:39.847757 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.847730    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "37bfe67c-8ca3-4ea1-916f-8012b9ec3195" (UID: "37bfe67c-8ca3-4ea1-916f-8012b9ec3195"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:21:39.946004 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.945964    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-model-cache\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.946004 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.945999    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-home\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.946004 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.946011    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-tls-certs\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.946267 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.946025    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2473cc6b-0bf7-42b7-8689-ccc6bd736912-dshm\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.946267 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.946036    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtjmx\" (UniqueName: \"kubernetes.io/projected/2473cc6b-0bf7-42b7-8689-ccc6bd736912-kube-api-access-wtjmx\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.946267 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.946050    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64462\" (UniqueName: \"kubernetes.io/projected/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-kube-api-access-64462\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.946267 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.946061    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37bfe67c-8ca3-4ea1-916f-8012b9ec3195-dshm\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.946267 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.946072    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2473cc6b-0bf7-42b7-8689-ccc6bd736912-tls-certs\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:21:39.966868 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:39.966828    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:40.320203 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.320168    2564 generic.go:356] "Generic (PLEG): container finished" podID="6702c4b7-5609-496b-808f-9a227938f6ad" containerID="b8d656a0639ea4b149472189a6d540330bc8b657c0de90590007285e62c8eafe" exitCode=0
Sep 15 21:21:40.320756 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.320229    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"
Sep 15 21:21:40.320756 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.320264    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn" event={"ID":"6702c4b7-5609-496b-808f-9a227938f6ad","Type":"ContainerDied","Data":"b8d656a0639ea4b149472189a6d540330bc8b657c0de90590007285e62c8eafe"}
Sep 15 21:21:40.320756 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.320326    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn" event={"ID":"6702c4b7-5609-496b-808f-9a227938f6ad","Type":"ContainerDied","Data":"2afa5913e48c771113341b6d99122c5db2cd869b2eaf3bb4bf2ad196aced6aad"}
Sep 15 21:21:40.320756 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.320355    2564 scope.go:117] "RemoveContainer" containerID="b8d656a0639ea4b149472189a6d540330bc8b657c0de90590007285e62c8eafe"
Sep 15 21:21:40.321950 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.321785    2564 generic.go:356] "Generic (PLEG): container finished" podID="2473cc6b-0bf7-42b7-8689-ccc6bd736912" containerID="2117e6189bc9996ac8ae015aeeb6192060c12f97270f8af271137d647b8d6491" exitCode=0
Sep 15 21:21:40.321950 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.321859    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"
Sep 15 21:21:40.321950 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.321872    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9" event={"ID":"2473cc6b-0bf7-42b7-8689-ccc6bd736912","Type":"ContainerDied","Data":"2117e6189bc9996ac8ae015aeeb6192060c12f97270f8af271137d647b8d6491"}
Sep 15 21:21:40.321950 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.321903    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9" event={"ID":"2473cc6b-0bf7-42b7-8689-ccc6bd736912","Type":"ContainerDied","Data":"ff2c4059286f37c766f58eef63eb842cb81d4649aba42d9b65833bedc2e48adf"}
Sep 15 21:21:40.323597 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.323572    2564 generic.go:356] "Generic (PLEG): container finished" podID="37bfe67c-8ca3-4ea1-916f-8012b9ec3195" containerID="f8c77b83856f1439774c439b5c19ea61aff299edf4c033dd39e15fd21f765d15" exitCode=0
Sep 15 21:21:40.323740 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.323653    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"
Sep 15 21:21:40.323740 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.323677    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg" event={"ID":"37bfe67c-8ca3-4ea1-916f-8012b9ec3195","Type":"ContainerDied","Data":"f8c77b83856f1439774c439b5c19ea61aff299edf4c033dd39e15fd21f765d15"}
Sep 15 21:21:40.323740 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.323710    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg" event={"ID":"37bfe67c-8ca3-4ea1-916f-8012b9ec3195","Type":"ContainerDied","Data":"c9d0d79ef1abf69b13afe64d726828359ca4b717e0cde3518c4b8259445017f6"}
Sep 15 21:21:40.325257 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.325235    2564 generic.go:356] "Generic (PLEG): container finished" podID="784be271-ba64-4757-9c89-cc238f0e3f22" containerID="f3e46b0c9a5346172d5dacba28fe9f7a08c98b8dc617acf58eba15441cea7d29" exitCode=0
Sep 15 21:21:40.325412 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.325281    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"
Sep 15 21:21:40.325412 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.325315    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g" event={"ID":"784be271-ba64-4757-9c89-cc238f0e3f22","Type":"ContainerDied","Data":"f3e46b0c9a5346172d5dacba28fe9f7a08c98b8dc617acf58eba15441cea7d29"}
Sep 15 21:21:40.325412 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.325348    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g" event={"ID":"784be271-ba64-4757-9c89-cc238f0e3f22","Type":"ContainerDied","Data":"50df71157557654fbc607487f90ca69fed8c2498f6858f15ec1fbdb02cec79df"}
Sep 15 21:21:40.331412 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.331395    2564 scope.go:117] "RemoveContainer" containerID="b8d656a0639ea4b149472189a6d540330bc8b657c0de90590007285e62c8eafe"
Sep 15 21:21:40.331706 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:21:40.331687    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8d656a0639ea4b149472189a6d540330bc8b657c0de90590007285e62c8eafe\": container with ID starting with b8d656a0639ea4b149472189a6d540330bc8b657c0de90590007285e62c8eafe not found: ID does not exist" containerID="b8d656a0639ea4b149472189a6d540330bc8b657c0de90590007285e62c8eafe"
Sep 15 21:21:40.331771 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.331718    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8d656a0639ea4b149472189a6d540330bc8b657c0de90590007285e62c8eafe"} err="failed to get container status \"b8d656a0639ea4b149472189a6d540330bc8b657c0de90590007285e62c8eafe\": rpc error: code = NotFound desc = could not find container \"b8d656a0639ea4b149472189a6d540330bc8b657c0de90590007285e62c8eafe\": container with ID starting with b8d656a0639ea4b149472189a6d540330bc8b657c0de90590007285e62c8eafe not found: ID does not exist"
Sep 15 21:21:40.331771 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.331736    2564 scope.go:117] "RemoveContainer" containerID="2117e6189bc9996ac8ae015aeeb6192060c12f97270f8af271137d647b8d6491"
Sep 15 21:21:40.340414 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.340394    2564 scope.go:117] "RemoveContainer" containerID="2117e6189bc9996ac8ae015aeeb6192060c12f97270f8af271137d647b8d6491"
Sep 15 21:21:40.340729 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:21:40.340709    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2117e6189bc9996ac8ae015aeeb6192060c12f97270f8af271137d647b8d6491\": container with ID starting with 2117e6189bc9996ac8ae015aeeb6192060c12f97270f8af271137d647b8d6491 not found: ID does not exist" containerID="2117e6189bc9996ac8ae015aeeb6192060c12f97270f8af271137d647b8d6491"
Sep 15 21:21:40.340808 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.340735    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2117e6189bc9996ac8ae015aeeb6192060c12f97270f8af271137d647b8d6491"} err="failed to get container status \"2117e6189bc9996ac8ae015aeeb6192060c12f97270f8af271137d647b8d6491\": rpc error: code = NotFound desc = could not find container \"2117e6189bc9996ac8ae015aeeb6192060c12f97270f8af271137d647b8d6491\": container with ID starting with 2117e6189bc9996ac8ae015aeeb6192060c12f97270f8af271137d647b8d6491 not found: ID does not exist"
Sep 15 21:21:40.340808 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.340752    2564 scope.go:117] "RemoveContainer" containerID="f8c77b83856f1439774c439b5c19ea61aff299edf4c033dd39e15fd21f765d15"
Sep 15 21:21:40.346392 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.346365    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"]
Sep 15 21:21:40.349253 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.349233    2564 scope.go:117] "RemoveContainer" containerID="f8c77b83856f1439774c439b5c19ea61aff299edf4c033dd39e15fd21f765d15"
Sep 15 21:21:40.349569 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:21:40.349547    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8c77b83856f1439774c439b5c19ea61aff299edf4c033dd39e15fd21f765d15\": container with ID starting with f8c77b83856f1439774c439b5c19ea61aff299edf4c033dd39e15fd21f765d15 not found: ID does not exist" containerID="f8c77b83856f1439774c439b5c19ea61aff299edf4c033dd39e15fd21f765d15"
Sep 15 21:21:40.349652 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.349576    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8c77b83856f1439774c439b5c19ea61aff299edf4c033dd39e15fd21f765d15"} err="failed to get container status \"f8c77b83856f1439774c439b5c19ea61aff299edf4c033dd39e15fd21f765d15\": rpc error: code = NotFound desc = could not find container \"f8c77b83856f1439774c439b5c19ea61aff299edf4c033dd39e15fd21f765d15\": container with ID starting with f8c77b83856f1439774c439b5c19ea61aff299edf4c033dd39e15fd21f765d15 not found: ID does not exist"
Sep 15 21:21:40.349652 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.349594    2564 scope.go:117] "RemoveContainer" containerID="f3e46b0c9a5346172d5dacba28fe9f7a08c98b8dc617acf58eba15441cea7d29"
Sep 15 21:21:40.350337 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.350313    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-544890/distinct-w0-544890-kserve-56489cd5cf-wthmg"]
Sep 15 21:21:40.357982 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.357965    2564 scope.go:117] "RemoveContainer" containerID="f3e46b0c9a5346172d5dacba28fe9f7a08c98b8dc617acf58eba15441cea7d29"
Sep 15 21:21:40.358303 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:21:40.358277    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3e46b0c9a5346172d5dacba28fe9f7a08c98b8dc617acf58eba15441cea7d29\": container with ID starting with f3e46b0c9a5346172d5dacba28fe9f7a08c98b8dc617acf58eba15441cea7d29 not found: ID does not exist" containerID="f3e46b0c9a5346172d5dacba28fe9f7a08c98b8dc617acf58eba15441cea7d29"
Sep 15 21:21:40.358404 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.358317    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3e46b0c9a5346172d5dacba28fe9f7a08c98b8dc617acf58eba15441cea7d29"} err="failed to get container status \"f3e46b0c9a5346172d5dacba28fe9f7a08c98b8dc617acf58eba15441cea7d29\": rpc error: code = NotFound desc = could not find container \"f3e46b0c9a5346172d5dacba28fe9f7a08c98b8dc617acf58eba15441cea7d29\": container with ID starting with f3e46b0c9a5346172d5dacba28fe9f7a08c98b8dc617acf58eba15441cea7d29 not found: ID does not exist"
Sep 15 21:21:40.360863 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.360841    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"]
Sep 15 21:21:40.364561 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.364541    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-544890/simulator-w0-544890-kserve-5fd7dd49-lvgdn"]
Sep 15 21:21:40.374672 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.374651    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"]
Sep 15 21:21:40.378175 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.378150    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-544890/distinct-2-w0-544890-kserve-6bfc5cd6d-jc94g"]
Sep 15 21:21:40.388999 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.388974    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"]
Sep 15 21:21:40.390803 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.390780    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-544890/embedding-w0-544890-kserve-5f8c78867b-2rvt9"]
Sep 15 21:21:40.678224 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:40.678186    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:41.468918 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:41.468882    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:41.565965 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:41.565938    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:41.671297 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:41.671251    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:41.773728 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:41.773629    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:41.929802 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:41.929755    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:41.973226 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:41.973191    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2473cc6b-0bf7-42b7-8689-ccc6bd736912" path="/var/lib/kubelet/pods/2473cc6b-0bf7-42b7-8689-ccc6bd736912/volumes"
Sep 15 21:21:41.973632 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:41.973616    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37bfe67c-8ca3-4ea1-916f-8012b9ec3195" path="/var/lib/kubelet/pods/37bfe67c-8ca3-4ea1-916f-8012b9ec3195/volumes"
Sep 15 21:21:41.973960 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:41.973948    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6702c4b7-5609-496b-808f-9a227938f6ad" path="/var/lib/kubelet/pods/6702c4b7-5609-496b-808f-9a227938f6ad/volumes"
Sep 15 21:21:41.974627 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:41.974612    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="784be271-ba64-4757-9c89-cc238f0e3f22" path="/var/lib/kubelet/pods/784be271-ba64-4757-9c89-cc238f0e3f22/volumes"
Sep 15 21:21:42.029858 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:42.029766    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:42.254995 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:42.254959    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:42.356478 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:42.356441    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:42.859696 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:42.859661    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:43.109346 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:43.109293    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:43.877441 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:43.877406    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:44.089882 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:44.089852    2564 scope.go:117] "RemoveContainer" containerID="ed5aef2f3cdb674d2237970a80c3c93470cca7d76c5669e82f45d11440c36dee"
Sep 15 21:21:45.808412 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:45.808377    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:45.942396 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:45.942363    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:46.938881 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:46.938838    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:47.737622 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:47.737588    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:48.829549 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:48.829510    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:49.698244 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:49.698200    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:50.455130 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.455093    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"]
Sep 15 21:21:50.455536 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.455465    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b27dd28b-9a41-4a3e-a821-12ad8fb7a413" containerName="maas-api"
Sep 15 21:21:50.455536 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.455476    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b27dd28b-9a41-4a3e-a821-12ad8fb7a413" containerName="maas-api"
Sep 15 21:21:50.455536 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.455505    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37bfe67c-8ca3-4ea1-916f-8012b9ec3195" containerName="main"
Sep 15 21:21:50.455536 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.455514    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="37bfe67c-8ca3-4ea1-916f-8012b9ec3195" containerName="main"
Sep 15 21:21:50.455536 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.455525    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6702c4b7-5609-496b-808f-9a227938f6ad" containerName="main"
Sep 15 21:21:50.455536 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.455530    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6702c4b7-5609-496b-808f-9a227938f6ad" containerName="main"
Sep 15 21:21:50.455728 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.455543    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="784be271-ba64-4757-9c89-cc238f0e3f22" containerName="main"
Sep 15 21:21:50.455728 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.455550    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="784be271-ba64-4757-9c89-cc238f0e3f22" containerName="main"
Sep 15 21:21:50.455728 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.455560    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8779115-0e1b-4b97-81c6-501c16ce4412" containerName="payload-pre-processing"
Sep 15 21:21:50.455728 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.455565    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8779115-0e1b-4b97-81c6-501c16ce4412" containerName="payload-pre-processing"
Sep 15 21:21:50.455728 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.455572    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5a525f4-a59e-4444-a998-453225e4f1e4" containerName="payload-processing"
Sep 15 21:21:50.455728 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.455577    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5a525f4-a59e-4444-a998-453225e4f1e4" containerName="payload-processing"
Sep 15 21:21:50.455728 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.455592    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2473cc6b-0bf7-42b7-8689-ccc6bd736912" containerName="main"
Sep 15 21:21:50.455728 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.455597    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2473cc6b-0bf7-42b7-8689-ccc6bd736912" containerName="main"
Sep 15 21:21:50.455728 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.455647    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="784be271-ba64-4757-9c89-cc238f0e3f22" containerName="main"
Sep 15 21:21:50.455728 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.455657    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6702c4b7-5609-496b-808f-9a227938f6ad" containerName="main"
Sep 15 21:21:50.455728 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.455664    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b27dd28b-9a41-4a3e-a821-12ad8fb7a413" containerName="maas-api"
Sep 15 21:21:50.455728 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.455669    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5a525f4-a59e-4444-a998-453225e4f1e4" containerName="payload-processing"
Sep 15 21:21:50.455728 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.455674    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8779115-0e1b-4b97-81c6-501c16ce4412" containerName="payload-pre-processing"
Sep 15 21:21:50.455728 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.455681    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2473cc6b-0bf7-42b7-8689-ccc6bd736912" containerName="main"
Sep 15 21:21:50.455728 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.455687    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="37bfe67c-8ca3-4ea1-916f-8012b9ec3195" containerName="main"
Sep 15 21:21:50.460247 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.460226    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"
Sep 15 21:21:50.462848 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.462823    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-beb398-serving-cert\""
Sep 15 21:21:50.468224 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.468045    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"]
Sep 15 21:21:50.526685 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.526646    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-openshift-service-ca\") pod \"maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h\" (UID: \"b0dc2727-91c8-408d-80ac-1a2fee23a4b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"
Sep 15 21:21:50.526685 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.526696    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-metrics-certs\") pod \"maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h\" (UID: \"b0dc2727-91c8-408d-80ac-1a2fee23a4b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"
Sep 15 21:21:50.526938 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.526849    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-maas-api-tls\") pod \"maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h\" (UID: \"b0dc2727-91c8-408d-80ac-1a2fee23a4b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"
Sep 15 21:21:50.526938 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.526926    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfpdf\" (UniqueName: \"kubernetes.io/projected/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-kube-api-access-cfpdf\") pod \"maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h\" (UID: \"b0dc2727-91c8-408d-80ac-1a2fee23a4b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"
Sep 15 21:21:50.546333 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.546285    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:50.627863 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.627822    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-openshift-service-ca\") pod \"maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h\" (UID: \"b0dc2727-91c8-408d-80ac-1a2fee23a4b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"
Sep 15 21:21:50.627863 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.627866    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-metrics-certs\") pod \"maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h\" (UID: \"b0dc2727-91c8-408d-80ac-1a2fee23a4b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"
Sep 15 21:21:50.628136 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.627939    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-maas-api-tls\") pod \"maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h\" (UID: \"b0dc2727-91c8-408d-80ac-1a2fee23a4b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"
Sep 15 21:21:50.628136 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.627993    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfpdf\" (UniqueName: \"kubernetes.io/projected/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-kube-api-access-cfpdf\") pod \"maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h\" (UID: \"b0dc2727-91c8-408d-80ac-1a2fee23a4b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"
Sep 15 21:21:50.628253 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:21:50.628150    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-serving-cert: secret "maas-api-e2e-api-b-beb398-serving-cert" not found
Sep 15 21:21:50.628302 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:21:50.628271    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-maas-api-tls podName:b0dc2727-91c8-408d-80ac-1a2fee23a4b4 nodeName:}" failed. No retries permitted until 2026-09-15 21:21:51.128246954 +0000 UTC m=+1447.670409583 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-maas-api-tls") pod "maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h" (UID: "b0dc2727-91c8-408d-80ac-1a2fee23a4b4") : secret "maas-api-e2e-api-b-beb398-serving-cert" not found
Sep 15 21:21:50.628693 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.628673    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-openshift-service-ca\") pod \"maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h\" (UID: \"b0dc2727-91c8-408d-80ac-1a2fee23a4b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"
Sep 15 21:21:50.630621 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.630602    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-metrics-certs\") pod \"maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h\" (UID: \"b0dc2727-91c8-408d-80ac-1a2fee23a4b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"
Sep 15 21:21:50.637272 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.637245    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfpdf\" (UniqueName: \"kubernetes.io/projected/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-kube-api-access-cfpdf\") pod \"maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h\" (UID: \"b0dc2727-91c8-408d-80ac-1a2fee23a4b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"
Sep 15 21:21:50.647838 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.647804    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:50.908896 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:50.908855    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:51.031014 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.030970    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:51.129354 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.129308    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:21:51.134246 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.134224    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-maas-api-tls\") pod \"maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h\" (UID: \"b0dc2727-91c8-408d-80ac-1a2fee23a4b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"
Sep 15 21:21:51.137275 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.137250    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-maas-api-tls\") pod \"maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h\" (UID: \"b0dc2727-91c8-408d-80ac-1a2fee23a4b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"
Sep 15 21:21:51.138080 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.138058    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb"]
Sep 15 21:21:51.141581 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.141564    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb"
Sep 15 21:21:51.145356 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.145330    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-beb398-dockercfg-k4zlb\""
Sep 15 21:21:51.145477 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.145358    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-beb398\""
Sep 15 21:21:51.157646 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.157617    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb"]
Sep 15 21:21:51.167401 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.167321    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp"]
Sep 15 21:21:51.170722 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.170697    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp"
Sep 15 21:21:51.183016 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.182986    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp"]
Sep 15 21:21:51.235384 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.235347    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a340b1d-152f-46a0-903a-278d9a2d1c69-plugins-config-volume\") pod \"payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp\" (UID: \"1a340b1d-152f-46a0-903a-278d9a2d1c69\") " pod="openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp"
Sep 15 21:21:51.235584 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.235402    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8753b91f-e06a-4170-b7b1-08bbce36f7c9-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb\" (UID: \"8753b91f-e06a-4170-b7b1-08bbce36f7c9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb"
Sep 15 21:21:51.235584 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.235476    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4btsh\" (UniqueName: \"kubernetes.io/projected/1a340b1d-152f-46a0-903a-278d9a2d1c69-kube-api-access-4btsh\") pod \"payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp\" (UID: \"1a340b1d-152f-46a0-903a-278d9a2d1c69\") " pod="openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp"
Sep 15 21:21:51.235584 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.235572    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdn9h\" (UniqueName: \"kubernetes.io/projected/8753b91f-e06a-4170-b7b1-08bbce36f7c9-kube-api-access-vdn9h\") pod \"payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb\" (UID: \"8753b91f-e06a-4170-b7b1-08bbce36f7c9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb"
Sep 15 21:21:51.336259 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.336219    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a340b1d-152f-46a0-903a-278d9a2d1c69-plugins-config-volume\") pod \"payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp\" (UID: \"1a340b1d-152f-46a0-903a-278d9a2d1c69\") " pod="openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp"
Sep 15 21:21:51.336439 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.336280    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8753b91f-e06a-4170-b7b1-08bbce36f7c9-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb\" (UID: \"8753b91f-e06a-4170-b7b1-08bbce36f7c9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb"
Sep 15 21:21:51.336439 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.336334    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4btsh\" (UniqueName: \"kubernetes.io/projected/1a340b1d-152f-46a0-903a-278d9a2d1c69-kube-api-access-4btsh\") pod \"payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp\" (UID: \"1a340b1d-152f-46a0-903a-278d9a2d1c69\") " pod="openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp"
Sep 15 21:21:51.336439 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.336385    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdn9h\" (UniqueName: \"kubernetes.io/projected/8753b91f-e06a-4170-b7b1-08bbce36f7c9-kube-api-access-vdn9h\") pod \"payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb\" (UID: \"8753b91f-e06a-4170-b7b1-08bbce36f7c9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb"
Sep 15 21:21:51.337052 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.337030    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a340b1d-152f-46a0-903a-278d9a2d1c69-plugins-config-volume\") pod \"payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp\" (UID: \"1a340b1d-152f-46a0-903a-278d9a2d1c69\") " pod="openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp"
Sep 15 21:21:51.337122 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.337049    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8753b91f-e06a-4170-b7b1-08bbce36f7c9-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb\" (UID: \"8753b91f-e06a-4170-b7b1-08bbce36f7c9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb"
Sep 15 21:21:51.345455 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.345418    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4btsh\" (UniqueName: \"kubernetes.io/projected/1a340b1d-152f-46a0-903a-278d9a2d1c69-kube-api-access-4btsh\") pod \"payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp\" (UID: \"1a340b1d-152f-46a0-903a-278d9a2d1c69\") " pod="openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp"
Sep 15 21:21:51.345588 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.345418    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdn9h\" (UniqueName: \"kubernetes.io/projected/8753b91f-e06a-4170-b7b1-08bbce36f7c9-kube-api-access-vdn9h\") pod \"payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb\" (UID: \"8753b91f-e06a-4170-b7b1-08bbce36f7c9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb"
Sep 15 21:21:51.373709 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.373668    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"
Sep 15 21:21:51.456379 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.456343    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb"
Sep 15 21:21:51.483944 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.483908    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp"
Sep 15 21:21:51.658072 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:21:51.658044    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0dc2727_91c8_408d_80ac_1a2fee23a4b4.slice/crio-9bd138ea3a607efce1432009f10da228d22e29201937542352067c6162a435dc WatchSource:0}: Error finding container 9bd138ea3a607efce1432009f10da228d22e29201937542352067c6162a435dc: Status 404 returned error can't find the container with id 9bd138ea3a607efce1432009f10da228d22e29201937542352067c6162a435dc
Sep 15 21:21:51.658734 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.658707    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"]
Sep 15 21:21:51.745266 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.745195    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb"]
Sep 15 21:21:51.747705 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:21:51.747680    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8753b91f_e06a_4170_b7b1_08bbce36f7c9.slice/crio-419273e0ed682db9fe56c54e479ba5a2813f91b7b6ce86832c309a660f914dbf WatchSource:0}: Error finding container 419273e0ed682db9fe56c54e479ba5a2813f91b7b6ce86832c309a660f914dbf: Status 404 returned error can't find the container with id 419273e0ed682db9fe56c54e479ba5a2813f91b7b6ce86832c309a660f914dbf
Sep 15 21:21:51.780997 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:51.780973    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp"]
Sep 15 21:21:51.786739 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:21:51.786699    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a340b1d_152f_46a0_903a_278d9a2d1c69.slice/crio-867c143a23a143531715a9454e9168314f1ea5a6789d638dd2ff6fb7efb181e4 WatchSource:0}: Error finding container 867c143a23a143531715a9454e9168314f1ea5a6789d638dd2ff6fb7efb181e4: Status 404 returned error can't find the container with id 867c143a23a143531715a9454e9168314f1ea5a6789d638dd2ff6fb7efb181e4
Sep 15 21:21:52.392334 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:52.392295    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp" event={"ID":"1a340b1d-152f-46a0-903a-278d9a2d1c69","Type":"ContainerStarted","Data":"ea458103a4dc8c65ad797c24e568a67c78e937da91d751b36a1c7c83ff56b3ee"}
Sep 15 21:21:52.392558 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:52.392357    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp" event={"ID":"1a340b1d-152f-46a0-903a-278d9a2d1c69","Type":"ContainerStarted","Data":"867c143a23a143531715a9454e9168314f1ea5a6789d638dd2ff6fb7efb181e4"}
Sep 15 21:21:52.392558 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:52.392399    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp"
Sep 15 21:21:52.394268 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:52.394244    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb" event={"ID":"8753b91f-e06a-4170-b7b1-08bbce36f7c9","Type":"ContainerStarted","Data":"e0b9aa7e8797db2842c357618810e7ec4af3a9ea5c0c807b2e9d0cc7cf33b106"}
Sep 15 21:21:52.394411 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:52.394286    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb" event={"ID":"8753b91f-e06a-4170-b7b1-08bbce36f7c9","Type":"ContainerStarted","Data":"419273e0ed682db9fe56c54e479ba5a2813f91b7b6ce86832c309a660f914dbf"}
Sep 15 21:21:52.394411 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:52.394321    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb"
Sep 15 21:21:52.396050 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:52.396025    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h" event={"ID":"b0dc2727-91c8-408d-80ac-1a2fee23a4b4","Type":"ContainerStarted","Data":"8dcefa7ac5614f57200ee607341871b63a8180d7ce097d728e51ea3ccd52a656"}
Sep 15 21:21:52.396149 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:52.396069    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h" event={"ID":"b0dc2727-91c8-408d-80ac-1a2fee23a4b4","Type":"ContainerStarted","Data":"9bd138ea3a607efce1432009f10da228d22e29201937542352067c6162a435dc"}
Sep 15 21:21:52.396204 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:52.396166    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"
Sep 15 21:21:52.412041 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:52.411987    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp" podStartSLOduration=1.41197259 podStartE2EDuration="1.41197259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:21:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:21:52.410914109 +0000 UTC m=+1448.953076767" watchObservedRunningTime="2026-09-15 21:21:52.41197259 +0000 UTC m=+1448.954135275"
Sep 15 21:21:52.432612 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:52.432550    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h" podStartSLOduration=2.110412159 podStartE2EDuration="2.432526718s" totalImagesPullingTime="322.114559ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:21:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:21:52.430815862 +0000 UTC m=+1448.972978521" watchObservedRunningTime="2026-09-15 21:21:52.432526718 +0000 UTC m=+1448.974689360"
Sep 15 21:21:52.478559 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:52.478474    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb" podStartSLOduration=1.478449893 podStartE2EDuration="1.478449893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:21:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:21:52.476943411 +0000 UTC m=+1449.019106055" watchObservedRunningTime="2026-09-15 21:21:52.478449893 +0000 UTC m=+1449.020612528"
Sep 15 21:21:58.408909 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:21:58.408880    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"
Sep 15 21:22:02.336665 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:02.336635    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:02.435757 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:02.435715    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:03.405828 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:03.405798    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb"
Sep 15 21:22:03.406245 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:03.405884    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp"
Sep 15 21:22:10.885216 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:10.885178    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:11.678314 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:11.678273    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:12.977018 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:12.976976    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:13.760785 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:13.760754    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:14.572375 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:14.572338    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:15.283822 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:15.283791    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:15.823382 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:15.823338    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:16.002163 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:16.002122    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:16.908671 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:16.908641    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:17.589336 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:17.589298    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:19.088197 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:19.088160    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:20.064046 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:20.064013    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:20.775794 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:20.775758    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:21.596221 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:21.596186    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:22.499915 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:22.499875    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:23.340863 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:23.340768    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:24.110012 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:24.109979    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:24.885524 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:24.885474    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:34.562215 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:34.562175    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:34.651261 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:34.651222    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:34.724553 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:34.724515    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:34.811894 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:34.811850    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:34.956993 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:34.956956    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:35.199550 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:35.199512    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:35.279374 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:35.279277    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:35.592406 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:35.592317    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:36.043996 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:36.043965    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:36.794114 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:36.794072    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:37.708780 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:37.708739    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:38.670624 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:38.670587    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:40.985752 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:40.985714    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:41.522878 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:41.522765    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:41.616315 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:41.616254    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:41.932678 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:41.932638    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:43.288840 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:43.288797    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:44.014388 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:44.014360    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-7t46m_c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf/console-operator/1.log"
Sep 15 21:22:44.018144 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:44.018112    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4c7k_fc205950-caf0-4c47-84ce-a8fc54a1afb2/ovn-acl-logging/0.log"
Sep 15 21:22:44.019627 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:44.019603    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-7t46m_c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf/console-operator/1.log"
Sep 15 21:22:44.023416 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:44.023388    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4c7k_fc205950-caf0-4c47-84ce-a8fc54a1afb2/ovn-acl-logging/0.log"
Sep 15 21:22:44.358326 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:44.358287    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:45.155019 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:45.154978    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:46.272905 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:46.272867    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:46.713551 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:46.713518    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:47.458029 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:47.457987    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:57.718320 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:57.718282    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"]
Sep 15 21:22:57.720987 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:57.718563    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h" podUID="b0dc2727-91c8-408d-80ac-1a2fee23a4b4" containerName="maas-api" containerID="cri-o://8dcefa7ac5614f57200ee607341871b63a8180d7ce097d728e51ea3ccd52a656" gracePeriod=30
Sep 15 21:22:57.748537 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:57.748501    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:57.781181 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:57.781141    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp"]
Sep 15 21:22:57.781575 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:57.781421    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp" podUID="1a340b1d-152f-46a0-903a-278d9a2d1c69" containerName="payload-processing" containerID="cri-o://ea458103a4dc8c65ad797c24e568a67c78e937da91d751b36a1c7c83ff56b3ee" gracePeriod=30
Sep 15 21:22:57.793651 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:57.793564    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb"]
Sep 15 21:22:57.793865 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:57.793831    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb" podUID="8753b91f-e06a-4170-b7b1-08bbce36f7c9" containerName="payload-pre-processing" containerID="cri-o://e0b9aa7e8797db2842c357618810e7ec4af3a9ea5c0c807b2e9d0cc7cf33b106" gracePeriod=30
Sep 15 21:22:57.826734 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:57.826697    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:57.893172 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:57.893140    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:57.976880 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:57.976741    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:58.039926 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.039867    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:58.080478 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.080450    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp"
Sep 15 21:22:58.083901 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.083872    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"
Sep 15 21:22:58.114334 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.114308    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb"
Sep 15 21:22:58.203686 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.203654    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfpdf\" (UniqueName: \"kubernetes.io/projected/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-kube-api-access-cfpdf\") pod \"b0dc2727-91c8-408d-80ac-1a2fee23a4b4\" (UID: \"b0dc2727-91c8-408d-80ac-1a2fee23a4b4\") "
Sep 15 21:22:58.203914 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.203715    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-openshift-service-ca\") pod \"b0dc2727-91c8-408d-80ac-1a2fee23a4b4\" (UID: \"b0dc2727-91c8-408d-80ac-1a2fee23a4b4\") "
Sep 15 21:22:58.203914 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.203733    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-maas-api-tls\") pod \"b0dc2727-91c8-408d-80ac-1a2fee23a4b4\" (UID: \"b0dc2727-91c8-408d-80ac-1a2fee23a4b4\") "
Sep 15 21:22:58.203914 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.203763    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a340b1d-152f-46a0-903a-278d9a2d1c69-plugins-config-volume\") pod \"1a340b1d-152f-46a0-903a-278d9a2d1c69\" (UID: \"1a340b1d-152f-46a0-903a-278d9a2d1c69\") "
Sep 15 21:22:58.203914 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.203807    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4btsh\" (UniqueName: \"kubernetes.io/projected/1a340b1d-152f-46a0-903a-278d9a2d1c69-kube-api-access-4btsh\") pod \"1a340b1d-152f-46a0-903a-278d9a2d1c69\" (UID: \"1a340b1d-152f-46a0-903a-278d9a2d1c69\") "
Sep 15 21:22:58.204171 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.204132    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b0dc2727-91c8-408d-80ac-1a2fee23a4b4" (UID: "b0dc2727-91c8-408d-80ac-1a2fee23a4b4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:22:58.204381 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.203840    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-metrics-certs\") pod \"b0dc2727-91c8-408d-80ac-1a2fee23a4b4\" (UID: \"b0dc2727-91c8-408d-80ac-1a2fee23a4b4\") "
Sep 15 21:22:58.204467 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.204180    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a340b1d-152f-46a0-903a-278d9a2d1c69-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1a340b1d-152f-46a0-903a-278d9a2d1c69" (UID: "1a340b1d-152f-46a0-903a-278d9a2d1c69"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:22:58.204790 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.204765    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-openshift-service-ca\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:22:58.204920 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.204795    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a340b1d-152f-46a0-903a-278d9a2d1c69-plugins-config-volume\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:22:58.206329 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.206300    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b0dc2727-91c8-408d-80ac-1a2fee23a4b4" (UID: "b0dc2727-91c8-408d-80ac-1a2fee23a4b4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:22:58.206476 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.206452    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-kube-api-access-cfpdf" (OuterVolumeSpecName: "kube-api-access-cfpdf") pod "b0dc2727-91c8-408d-80ac-1a2fee23a4b4" (UID: "b0dc2727-91c8-408d-80ac-1a2fee23a4b4"). InnerVolumeSpecName "kube-api-access-cfpdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:22:58.206774 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.206742    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a340b1d-152f-46a0-903a-278d9a2d1c69-kube-api-access-4btsh" (OuterVolumeSpecName: "kube-api-access-4btsh") pod "1a340b1d-152f-46a0-903a-278d9a2d1c69" (UID: "1a340b1d-152f-46a0-903a-278d9a2d1c69"). InnerVolumeSpecName "kube-api-access-4btsh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:22:58.206859 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.206787    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b0dc2727-91c8-408d-80ac-1a2fee23a4b4" (UID: "b0dc2727-91c8-408d-80ac-1a2fee23a4b4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:22:58.306091 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.306003    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdn9h\" (UniqueName: \"kubernetes.io/projected/8753b91f-e06a-4170-b7b1-08bbce36f7c9-kube-api-access-vdn9h\") pod \"8753b91f-e06a-4170-b7b1-08bbce36f7c9\" (UID: \"8753b91f-e06a-4170-b7b1-08bbce36f7c9\") "
Sep 15 21:22:58.306241 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.306168    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8753b91f-e06a-4170-b7b1-08bbce36f7c9-plugins-config-volume\") pod \"8753b91f-e06a-4170-b7b1-08bbce36f7c9\" (UID: \"8753b91f-e06a-4170-b7b1-08bbce36f7c9\") "
Sep 15 21:22:58.306443 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.306418    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfpdf\" (UniqueName: \"kubernetes.io/projected/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-kube-api-access-cfpdf\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:22:58.306443 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.306441    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-maas-api-tls\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:22:58.306628 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.306455    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4btsh\" (UniqueName: \"kubernetes.io/projected/1a340b1d-152f-46a0-903a-278d9a2d1c69-kube-api-access-4btsh\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:22:58.306628 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.306466    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0dc2727-91c8-408d-80ac-1a2fee23a4b4-metrics-certs\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:22:58.306628 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.306556    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8753b91f-e06a-4170-b7b1-08bbce36f7c9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8753b91f-e06a-4170-b7b1-08bbce36f7c9" (UID: "8753b91f-e06a-4170-b7b1-08bbce36f7c9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:22:58.308249 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.308222    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8753b91f-e06a-4170-b7b1-08bbce36f7c9-kube-api-access-vdn9h" (OuterVolumeSpecName: "kube-api-access-vdn9h") pod "8753b91f-e06a-4170-b7b1-08bbce36f7c9" (UID: "8753b91f-e06a-4170-b7b1-08bbce36f7c9"). InnerVolumeSpecName "kube-api-access-vdn9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:22:58.328115 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.328089    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:58.407860 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.407826    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8753b91f-e06a-4170-b7b1-08bbce36f7c9-plugins-config-volume\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:22:58.407860 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.407856    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdn9h\" (UniqueName: \"kubernetes.io/projected/8753b91f-e06a-4170-b7b1-08bbce36f7c9-kube-api-access-vdn9h\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:22:58.429269 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.429230    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:58.757792 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.757756    2564 generic.go:356] "Generic (PLEG): container finished" podID="b0dc2727-91c8-408d-80ac-1a2fee23a4b4" containerID="8dcefa7ac5614f57200ee607341871b63a8180d7ce097d728e51ea3ccd52a656" exitCode=0
Sep 15 21:22:58.758209 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.757814    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"
Sep 15 21:22:58.758209 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.757849    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h" event={"ID":"b0dc2727-91c8-408d-80ac-1a2fee23a4b4","Type":"ContainerDied","Data":"8dcefa7ac5614f57200ee607341871b63a8180d7ce097d728e51ea3ccd52a656"}
Sep 15 21:22:58.758209 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.757939    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h" event={"ID":"b0dc2727-91c8-408d-80ac-1a2fee23a4b4","Type":"ContainerDied","Data":"9bd138ea3a607efce1432009f10da228d22e29201937542352067c6162a435dc"}
Sep 15 21:22:58.758209 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.757969    2564 scope.go:117] "RemoveContainer" containerID="8dcefa7ac5614f57200ee607341871b63a8180d7ce097d728e51ea3ccd52a656"
Sep 15 21:22:58.759393 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.759354    2564 generic.go:356] "Generic (PLEG): container finished" podID="1a340b1d-152f-46a0-903a-278d9a2d1c69" containerID="ea458103a4dc8c65ad797c24e568a67c78e937da91d751b36a1c7c83ff56b3ee" exitCode=0
Sep 15 21:22:58.759444 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.759418    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp"
Sep 15 21:22:58.759507 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.759441    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp" event={"ID":"1a340b1d-152f-46a0-903a-278d9a2d1c69","Type":"ContainerDied","Data":"ea458103a4dc8c65ad797c24e568a67c78e937da91d751b36a1c7c83ff56b3ee"}
Sep 15 21:22:58.759507 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.759481    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp" event={"ID":"1a340b1d-152f-46a0-903a-278d9a2d1c69","Type":"ContainerDied","Data":"867c143a23a143531715a9454e9168314f1ea5a6789d638dd2ff6fb7efb181e4"}
Sep 15 21:22:58.760783 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.760765    2564 generic.go:356] "Generic (PLEG): container finished" podID="8753b91f-e06a-4170-b7b1-08bbce36f7c9" containerID="e0b9aa7e8797db2842c357618810e7ec4af3a9ea5c0c807b2e9d0cc7cf33b106" exitCode=0
Sep 15 21:22:58.760871 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.760794    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb" event={"ID":"8753b91f-e06a-4170-b7b1-08bbce36f7c9","Type":"ContainerDied","Data":"e0b9aa7e8797db2842c357618810e7ec4af3a9ea5c0c807b2e9d0cc7cf33b106"}
Sep 15 21:22:58.760871 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.760810    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb"
Sep 15 21:22:58.760871 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.760821    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb" event={"ID":"8753b91f-e06a-4170-b7b1-08bbce36f7c9","Type":"ContainerDied","Data":"419273e0ed682db9fe56c54e479ba5a2813f91b7b6ce86832c309a660f914dbf"}
Sep 15 21:22:58.769704 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.769375    2564 scope.go:117] "RemoveContainer" containerID="8dcefa7ac5614f57200ee607341871b63a8180d7ce097d728e51ea3ccd52a656"
Sep 15 21:22:58.769847 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:22:58.769822    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dcefa7ac5614f57200ee607341871b63a8180d7ce097d728e51ea3ccd52a656\": container with ID starting with 8dcefa7ac5614f57200ee607341871b63a8180d7ce097d728e51ea3ccd52a656 not found: ID does not exist" containerID="8dcefa7ac5614f57200ee607341871b63a8180d7ce097d728e51ea3ccd52a656"
Sep 15 21:22:58.769917 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.769859    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dcefa7ac5614f57200ee607341871b63a8180d7ce097d728e51ea3ccd52a656"} err="failed to get container status \"8dcefa7ac5614f57200ee607341871b63a8180d7ce097d728e51ea3ccd52a656\": rpc error: code = NotFound desc = could not find container \"8dcefa7ac5614f57200ee607341871b63a8180d7ce097d728e51ea3ccd52a656\": container with ID starting with 8dcefa7ac5614f57200ee607341871b63a8180d7ce097d728e51ea3ccd52a656 not found: ID does not exist"
Sep 15 21:22:58.769917 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.769885    2564 scope.go:117] "RemoveContainer" containerID="ea458103a4dc8c65ad797c24e568a67c78e937da91d751b36a1c7c83ff56b3ee"
Sep 15 21:22:58.778757 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.778739    2564 scope.go:117] "RemoveContainer" containerID="ea458103a4dc8c65ad797c24e568a67c78e937da91d751b36a1c7c83ff56b3ee"
Sep 15 21:22:58.779029 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:22:58.779010    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea458103a4dc8c65ad797c24e568a67c78e937da91d751b36a1c7c83ff56b3ee\": container with ID starting with ea458103a4dc8c65ad797c24e568a67c78e937da91d751b36a1c7c83ff56b3ee not found: ID does not exist" containerID="ea458103a4dc8c65ad797c24e568a67c78e937da91d751b36a1c7c83ff56b3ee"
Sep 15 21:22:58.779076 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.779038    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea458103a4dc8c65ad797c24e568a67c78e937da91d751b36a1c7c83ff56b3ee"} err="failed to get container status \"ea458103a4dc8c65ad797c24e568a67c78e937da91d751b36a1c7c83ff56b3ee\": rpc error: code = NotFound desc = could not find container \"ea458103a4dc8c65ad797c24e568a67c78e937da91d751b36a1c7c83ff56b3ee\": container with ID starting with ea458103a4dc8c65ad797c24e568a67c78e937da91d751b36a1c7c83ff56b3ee not found: ID does not exist"
Sep 15 21:22:58.779076 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.779056    2564 scope.go:117] "RemoveContainer" containerID="e0b9aa7e8797db2842c357618810e7ec4af3a9ea5c0c807b2e9d0cc7cf33b106"
Sep 15 21:22:58.785961 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.785928    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"]
Sep 15 21:22:58.788483 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.788254    2564 scope.go:117] "RemoveContainer" containerID="e0b9aa7e8797db2842c357618810e7ec4af3a9ea5c0c807b2e9d0cc7cf33b106"
Sep 15 21:22:58.788679 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:22:58.788647    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0b9aa7e8797db2842c357618810e7ec4af3a9ea5c0c807b2e9d0cc7cf33b106\": container with ID starting with e0b9aa7e8797db2842c357618810e7ec4af3a9ea5c0c807b2e9d0cc7cf33b106 not found: ID does not exist" containerID="e0b9aa7e8797db2842c357618810e7ec4af3a9ea5c0c807b2e9d0cc7cf33b106"
Sep 15 21:22:58.788748 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.788692    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0b9aa7e8797db2842c357618810e7ec4af3a9ea5c0c807b2e9d0cc7cf33b106"} err="failed to get container status \"e0b9aa7e8797db2842c357618810e7ec4af3a9ea5c0c807b2e9d0cc7cf33b106\": rpc error: code = NotFound desc = could not find container \"e0b9aa7e8797db2842c357618810e7ec4af3a9ea5c0c807b2e9d0cc7cf33b106\": container with ID starting with e0b9aa7e8797db2842c357618810e7ec4af3a9ea5c0c807b2e9d0cc7cf33b106 not found: ID does not exist"
Sep 15 21:22:58.790479 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.790453    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-beb398-66b7649bc9-j5h4h"]
Sep 15 21:22:58.803209 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.803180    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp"]
Sep 15 21:22:58.808982 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.808951    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-beb398-65f7cb8c59-kdlcp"]
Sep 15 21:22:58.819870 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.819839    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb"]
Sep 15 21:22:58.824135 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:58.824109    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-beb398-7cb868fc8b-6jvmb"]
Sep 15 21:22:59.400073 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:59.400034    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:22:59.972881 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:59.972848    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a340b1d-152f-46a0-903a-278d9a2d1c69" path="/var/lib/kubelet/pods/1a340b1d-152f-46a0-903a-278d9a2d1c69/volumes"
Sep 15 21:22:59.973253 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:59.973224    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8753b91f-e06a-4170-b7b1-08bbce36f7c9" path="/var/lib/kubelet/pods/8753b91f-e06a-4170-b7b1-08bbce36f7c9/volumes"
Sep 15 21:22:59.973578 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:22:59.973564    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0dc2727-91c8-408d-80ac-1a2fee23a4b4" path="/var/lib/kubelet/pods/b0dc2727-91c8-408d-80ac-1a2fee23a4b4/volumes"
Sep 15 21:23:00.366029 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:00.365995    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:01.431847 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:01.431796    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:02.497194 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:02.497160    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:05.467110 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:05.467025    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:05.579013 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:05.578455    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:05.817665 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:05.817569    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:06.453202 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:06.453159    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:07.455899 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:07.455850    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:08.491713 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:08.491680    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:09.418519 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:09.418466    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:10.278217 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:10.278184    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:11.079229 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:11.079188    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:12.292378 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:12.292340    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:13.400887 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:13.400852    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:16.609216 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:16.609182    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:16.676077 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:16.676039    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:16.736732 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:16.736692    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:16.801248 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:16.801208    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:16.869323 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:16.869229    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:17.151175 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:17.151068    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:17.250309 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:17.250274    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:18.386137 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:18.386106    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:18.976812 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:18.976778    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:19.668707 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:19.668675    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:22.752228 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:22.752193    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:22.854727 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:22.854684    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:23.018800 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:23.018699    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:23.490244 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:23.490197    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:24.219342 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:24.219308    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:24.892244 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:24.892205    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:25.087964 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.087922    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"]
Sep 15 21:23:25.088506 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.088471    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a340b1d-152f-46a0-903a-278d9a2d1c69" containerName="payload-processing"
Sep 15 21:23:25.088628 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.088512    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a340b1d-152f-46a0-903a-278d9a2d1c69" containerName="payload-processing"
Sep 15 21:23:25.088628 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.088531    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0dc2727-91c8-408d-80ac-1a2fee23a4b4" containerName="maas-api"
Sep 15 21:23:25.088628 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.088546    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0dc2727-91c8-408d-80ac-1a2fee23a4b4" containerName="maas-api"
Sep 15 21:23:25.088628 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.088575    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8753b91f-e06a-4170-b7b1-08bbce36f7c9" containerName="payload-pre-processing"
Sep 15 21:23:25.088628 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.088584    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8753b91f-e06a-4170-b7b1-08bbce36f7c9" containerName="payload-pre-processing"
Sep 15 21:23:25.088816 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.088682    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0dc2727-91c8-408d-80ac-1a2fee23a4b4" containerName="maas-api"
Sep 15 21:23:25.088816 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.088692    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a340b1d-152f-46a0-903a-278d9a2d1c69" containerName="payload-processing"
Sep 15 21:23:25.088816 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.088701    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8753b91f-e06a-4170-b7b1-08bbce36f7c9" containerName="payload-pre-processing"
Sep 15 21:23:25.091987 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.091970    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:25.095862 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.095833    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-2f34ef\"/\"openshift-service-ca.crt\""
Sep 15 21:23:25.095981 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.095868    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-2f34ef\"/\"kube-root-ca.crt\""
Sep 15 21:23:25.095981 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.095896    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-2f34ef\"/\"sub-test-model-2f34ef-kserve-self-signed-certs\""
Sep 15 21:23:25.095981 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.095833    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-2f34ef\"/\"default-dockercfg-fxdnv\""
Sep 15 21:23:25.101927 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.101901    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"]
Sep 15 21:23:25.109832 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.109787    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:25.147008 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.146908    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/270f505a-2dba-4a23-8a35-ebecd9741656-tls-certs\") pod \"sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") " pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:25.147008 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.146979    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-dshm\") pod \"sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") " pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:25.147251 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.147022    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-home\") pod \"sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") " pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:25.147251 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.147079    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgmjk\" (UniqueName: \"kubernetes.io/projected/270f505a-2dba-4a23-8a35-ebecd9741656-kube-api-access-hgmjk\") pod \"sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") " pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:25.147251 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.147120    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-model-cache\") pod \"sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") " pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:25.147251 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.147152    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-tmp-dir\") pod \"sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") " pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:25.248002 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.247969    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-home\") pod \"sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") " pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:25.248525 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.248022    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgmjk\" (UniqueName: \"kubernetes.io/projected/270f505a-2dba-4a23-8a35-ebecd9741656-kube-api-access-hgmjk\") pod \"sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") " pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:25.248525 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.248046    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-model-cache\") pod \"sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") " pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:25.248525 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.248072    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-tmp-dir\") pod \"sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") " pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:25.248525 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.248092    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/270f505a-2dba-4a23-8a35-ebecd9741656-tls-certs\") pod \"sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") " pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:25.248525 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.248130    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-dshm\") pod \"sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") " pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:25.248525 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.248461    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-home\") pod \"sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") " pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:25.248780 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.248527    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-model-cache\") pod \"sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") " pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:25.248780 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.248583    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-tmp-dir\") pod \"sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") " pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:25.250733 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.250703    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-dshm\") pod \"sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") " pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:25.250847 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.250764    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/270f505a-2dba-4a23-8a35-ebecd9741656-tls-certs\") pod \"sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") " pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:25.256224 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.256200    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgmjk\" (UniqueName: \"kubernetes.io/projected/270f505a-2dba-4a23-8a35-ebecd9741656-kube-api-access-hgmjk\") pod \"sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") " pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:25.402358 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.402264    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:25.676606 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.676582    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"]
Sep 15 21:23:25.678474 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:23:25.678450    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod270f505a_2dba_4a23_8a35_ebecd9741656.slice/crio-cd795115ed543c29c0bf81719d79346f5edfeaef966ecca926c9c7dc4f426a2a WatchSource:0}: Error finding container cd795115ed543c29c0bf81719d79346f5edfeaef966ecca926c9c7dc4f426a2a: Status 404 returned error can't find the container with id cd795115ed543c29c0bf81719d79346f5edfeaef966ecca926c9c7dc4f426a2a
Sep 15 21:23:25.845411 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.845369    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:25.905683 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.905631    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7" event={"ID":"270f505a-2dba-4a23-8a35-ebecd9741656","Type":"ContainerStarted","Data":"62fcb2352206a08b18a94b250535baab3eb638ab80249d2b5d20095744b67705"}
Sep 15 21:23:25.905683 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.905689    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7" event={"ID":"270f505a-2dba-4a23-8a35-ebecd9741656","Type":"ContainerStarted","Data":"cd795115ed543c29c0bf81719d79346f5edfeaef966ecca926c9c7dc4f426a2a"}
Sep 15 21:23:25.924150 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:25.924100    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7" podStartSLOduration=0.924085109 podStartE2EDuration="924.085109ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:23:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:23:25.921911934 +0000 UTC m=+1542.464074610" watchObservedRunningTime="2026-09-15 21:23:25.924085109 +0000 UTC m=+1542.466247744"
Sep 15 21:23:26.545938 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:26.545905    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:33.894916 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:33.894880    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:33.964887 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:33.964841    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:34.060326 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:34.060263    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:34.121556 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:34.121514    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:34.417255 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:34.417223    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:34.498419 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:34.498382    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:35.334619 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:35.334583    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:35.403448 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:35.403404    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:35.403448 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:35.403452    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:35.416330 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:35.416299    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:35.972729 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:35.972693    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:23:36.175299 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:36.175253    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:40.134220 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:40.134184    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:40.234802 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:40.234763    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:40.375358 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:40.375324    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:23:51.391318 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:23:51.391281    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:24:49.604306 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:24:49.604230    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"]
Sep 15 21:24:49.624451 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:24:49.624417    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:24:50.544193 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:24:50.544145    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:24:52.253096 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:24:52.253049    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:25:01.543154 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:01.543119    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:25:04.624110 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:04.624062    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7" podUID="270f505a-2dba-4a23-8a35-ebecd9741656" containerName="main" containerID="cri-o://62fcb2352206a08b18a94b250535baab3eb638ab80249d2b5d20095744b67705" gracePeriod=45
Sep 15 21:25:04.886142 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:04.886071    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:25:04.922005 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:04.921973    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-dshm\") pod \"270f505a-2dba-4a23-8a35-ebecd9741656\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") "
Sep 15 21:25:04.922203 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:04.922065    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgmjk\" (UniqueName: \"kubernetes.io/projected/270f505a-2dba-4a23-8a35-ebecd9741656-kube-api-access-hgmjk\") pod \"270f505a-2dba-4a23-8a35-ebecd9741656\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") "
Sep 15 21:25:04.922203 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:04.922093    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-tmp-dir\") pod \"270f505a-2dba-4a23-8a35-ebecd9741656\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") "
Sep 15 21:25:04.922203 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:04.922109    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/270f505a-2dba-4a23-8a35-ebecd9741656-tls-certs\") pod \"270f505a-2dba-4a23-8a35-ebecd9741656\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") "
Sep 15 21:25:04.922203 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:04.922128    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-model-cache\") pod \"270f505a-2dba-4a23-8a35-ebecd9741656\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") "
Sep 15 21:25:04.922203 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:04.922164    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-home\") pod \"270f505a-2dba-4a23-8a35-ebecd9741656\" (UID: \"270f505a-2dba-4a23-8a35-ebecd9741656\") "
Sep 15 21:25:04.922481 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:04.922425    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "270f505a-2dba-4a23-8a35-ebecd9741656" (UID: "270f505a-2dba-4a23-8a35-ebecd9741656"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:25:04.922481 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:04.922439    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-model-cache" (OuterVolumeSpecName: "model-cache") pod "270f505a-2dba-4a23-8a35-ebecd9741656" (UID: "270f505a-2dba-4a23-8a35-ebecd9741656"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:25:04.922612 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:04.922599    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-home" (OuterVolumeSpecName: "home") pod "270f505a-2dba-4a23-8a35-ebecd9741656" (UID: "270f505a-2dba-4a23-8a35-ebecd9741656"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:25:04.924582 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:04.924546    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/270f505a-2dba-4a23-8a35-ebecd9741656-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "270f505a-2dba-4a23-8a35-ebecd9741656" (UID: "270f505a-2dba-4a23-8a35-ebecd9741656"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:25:04.924715 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:04.924597    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-dshm" (OuterVolumeSpecName: "dshm") pod "270f505a-2dba-4a23-8a35-ebecd9741656" (UID: "270f505a-2dba-4a23-8a35-ebecd9741656"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:25:04.924715 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:04.924598    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/270f505a-2dba-4a23-8a35-ebecd9741656-kube-api-access-hgmjk" (OuterVolumeSpecName: "kube-api-access-hgmjk") pod "270f505a-2dba-4a23-8a35-ebecd9741656" (UID: "270f505a-2dba-4a23-8a35-ebecd9741656"). InnerVolumeSpecName "kube-api-access-hgmjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:25:05.022841 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:05.022790    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgmjk\" (UniqueName: \"kubernetes.io/projected/270f505a-2dba-4a23-8a35-ebecd9741656-kube-api-access-hgmjk\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:25:05.022841 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:05.022838    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-tmp-dir\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:25:05.022841 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:05.022853    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/270f505a-2dba-4a23-8a35-ebecd9741656-tls-certs\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:25:05.023096 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:05.022865    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-model-cache\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:25:05.023096 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:05.022876    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-home\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:25:05.023096 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:05.022888    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/270f505a-2dba-4a23-8a35-ebecd9741656-dshm\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:25:05.404873 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:05.404838    2564 generic.go:356] "Generic (PLEG): container finished" podID="270f505a-2dba-4a23-8a35-ebecd9741656" containerID="62fcb2352206a08b18a94b250535baab3eb638ab80249d2b5d20095744b67705" exitCode=0
Sep 15 21:25:05.405074 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:05.404946    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7" event={"ID":"270f505a-2dba-4a23-8a35-ebecd9741656","Type":"ContainerDied","Data":"62fcb2352206a08b18a94b250535baab3eb638ab80249d2b5d20095744b67705"}
Sep 15 21:25:05.405074 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:05.404955    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"
Sep 15 21:25:05.405074 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:05.404991    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7" event={"ID":"270f505a-2dba-4a23-8a35-ebecd9741656","Type":"ContainerDied","Data":"cd795115ed543c29c0bf81719d79346f5edfeaef966ecca926c9c7dc4f426a2a"}
Sep 15 21:25:05.405074 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:05.405014    2564 scope.go:117] "RemoveContainer" containerID="62fcb2352206a08b18a94b250535baab3eb638ab80249d2b5d20095744b67705"
Sep 15 21:25:05.414460 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:05.414440    2564 scope.go:117] "RemoveContainer" containerID="62fcb2352206a08b18a94b250535baab3eb638ab80249d2b5d20095744b67705"
Sep 15 21:25:05.414839 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:25:05.414819    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62fcb2352206a08b18a94b250535baab3eb638ab80249d2b5d20095744b67705\": container with ID starting with 62fcb2352206a08b18a94b250535baab3eb638ab80249d2b5d20095744b67705 not found: ID does not exist" containerID="62fcb2352206a08b18a94b250535baab3eb638ab80249d2b5d20095744b67705"
Sep 15 21:25:05.414902 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:05.414849    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62fcb2352206a08b18a94b250535baab3eb638ab80249d2b5d20095744b67705"} err="failed to get container status \"62fcb2352206a08b18a94b250535baab3eb638ab80249d2b5d20095744b67705\": rpc error: code = NotFound desc = could not find container \"62fcb2352206a08b18a94b250535baab3eb638ab80249d2b5d20095744b67705\": container with ID starting with 62fcb2352206a08b18a94b250535baab3eb638ab80249d2b5d20095744b67705 not found: ID does not exist"
Sep 15 21:25:05.427613 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:05.427582    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"]
Sep 15 21:25:05.430342 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:05.430309    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2f34ef/sub-test-model-2f34ef-kserve-7bbbcbfbff-g59w7"]
Sep 15 21:25:05.972706 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:05.972673    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="270f505a-2dba-4a23-8a35-ebecd9741656" path="/var/lib/kubelet/pods/270f505a-2dba-4a23-8a35-ebecd9741656/volumes"
Sep 15 21:25:26.768209 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:26.768167    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:25:26.868454 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:26.868418    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:25:30.761568 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:25:30.761515    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:06.629119 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:06.629018    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:07.644708 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:07.644666    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:08.112201 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:08.112167    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:08.466748 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:08.466648    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:08.896423 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:08.896390    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:10.791539 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:10.791504    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:11.405564 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:11.405517    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:11.497692 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:11.497655    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:11.783439 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:11.783331    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:12.408066 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:12.408027    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:13.164382 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.164349    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"]
Sep 15 21:26:13.164830 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.164814    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="270f505a-2dba-4a23-8a35-ebecd9741656" containerName="main"
Sep 15 21:26:13.164887 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.164833    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="270f505a-2dba-4a23-8a35-ebecd9741656" containerName="main"
Sep 15 21:26:13.164924 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.164895    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="270f505a-2dba-4a23-8a35-ebecd9741656" containerName="main"
Sep 15 21:26:13.168007 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.167980    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"
Sep 15 21:26:13.170787 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.170764    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-b8f396-serving-cert\""
Sep 15 21:26:13.179519 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.179473    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"]
Sep 15 21:26:13.202414 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.202382    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:13.281836 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.281798    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7bzf\" (UniqueName: \"kubernetes.io/projected/f4f1ff0a-c012-494a-8982-0ccefe32e543-kube-api-access-m7bzf\") pod \"maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5\" (UID: \"f4f1ff0a-c012-494a-8982-0ccefe32e543\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"
Sep 15 21:26:13.281836 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.281853    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4f1ff0a-c012-494a-8982-0ccefe32e543-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5\" (UID: \"f4f1ff0a-c012-494a-8982-0ccefe32e543\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"
Sep 15 21:26:13.282132 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.281957    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4f1ff0a-c012-494a-8982-0ccefe32e543-maas-api-tls\") pod \"maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5\" (UID: \"f4f1ff0a-c012-494a-8982-0ccefe32e543\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"
Sep 15 21:26:13.282132 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.282049    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f1ff0a-c012-494a-8982-0ccefe32e543-metrics-certs\") pod \"maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5\" (UID: \"f4f1ff0a-c012-494a-8982-0ccefe32e543\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"
Sep 15 21:26:13.303660 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.303623    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:13.383226 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.383188    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7bzf\" (UniqueName: \"kubernetes.io/projected/f4f1ff0a-c012-494a-8982-0ccefe32e543-kube-api-access-m7bzf\") pod \"maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5\" (UID: \"f4f1ff0a-c012-494a-8982-0ccefe32e543\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"
Sep 15 21:26:13.383226 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.383242    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4f1ff0a-c012-494a-8982-0ccefe32e543-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5\" (UID: \"f4f1ff0a-c012-494a-8982-0ccefe32e543\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"
Sep 15 21:26:13.383477 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.383272    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4f1ff0a-c012-494a-8982-0ccefe32e543-maas-api-tls\") pod \"maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5\" (UID: \"f4f1ff0a-c012-494a-8982-0ccefe32e543\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"
Sep 15 21:26:13.383477 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.383327    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f1ff0a-c012-494a-8982-0ccefe32e543-metrics-certs\") pod \"maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5\" (UID: \"f4f1ff0a-c012-494a-8982-0ccefe32e543\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"
Sep 15 21:26:13.384073 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.384031    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4f1ff0a-c012-494a-8982-0ccefe32e543-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5\" (UID: \"f4f1ff0a-c012-494a-8982-0ccefe32e543\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"
Sep 15 21:26:13.386012 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.385992    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4f1ff0a-c012-494a-8982-0ccefe32e543-maas-api-tls\") pod \"maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5\" (UID: \"f4f1ff0a-c012-494a-8982-0ccefe32e543\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"
Sep 15 21:26:13.386157 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.386136    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f1ff0a-c012-494a-8982-0ccefe32e543-metrics-certs\") pod \"maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5\" (UID: \"f4f1ff0a-c012-494a-8982-0ccefe32e543\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"
Sep 15 21:26:13.392003 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.391972    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7bzf\" (UniqueName: \"kubernetes.io/projected/f4f1ff0a-c012-494a-8982-0ccefe32e543-kube-api-access-m7bzf\") pod \"maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5\" (UID: \"f4f1ff0a-c012-494a-8982-0ccefe32e543\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"
Sep 15 21:26:13.413407 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.413373    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:13.482776 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.482679    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"
Sep 15 21:26:13.764454 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.764418    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"]
Sep 15 21:26:13.766132 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:26:13.766103    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4f1ff0a_c012_494a_8982_0ccefe32e543.slice/crio-2d117aa7a41602ea2c6718580c2317e597b83c6db7c801fb2d73cc345bce664e WatchSource:0}: Error finding container 2d117aa7a41602ea2c6718580c2317e597b83c6db7c801fb2d73cc345bce664e: Status 404 returned error can't find the container with id 2d117aa7a41602ea2c6718580c2317e597b83c6db7c801fb2d73cc345bce664e
Sep 15 21:26:13.767636 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.767613    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 21:26:13.850337 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.850302    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:13.938961 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.938914    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:13.973868 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.973836    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk"]
Sep 15 21:26:13.977133 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.977113    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk"
Sep 15 21:26:13.981137 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.981103    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-b8f396\""
Sep 15 21:26:13.981281 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.981104    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-b8f396-dockercfg-62zps\""
Sep 15 21:26:13.987765 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:13.987737    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk"]
Sep 15 21:26:14.090633 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:14.090545    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b08cf44-9dc1-47e3-974f-ee121b15559e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk\" (UID: \"7b08cf44-9dc1-47e3-974f-ee121b15559e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk"
Sep 15 21:26:14.090633 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:14.090589    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92mlk\" (UniqueName: \"kubernetes.io/projected/7b08cf44-9dc1-47e3-974f-ee121b15559e-kube-api-access-92mlk\") pod \"payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk\" (UID: \"7b08cf44-9dc1-47e3-974f-ee121b15559e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk"
Sep 15 21:26:14.192087 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:14.192047    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92mlk\" (UniqueName: \"kubernetes.io/projected/7b08cf44-9dc1-47e3-974f-ee121b15559e-kube-api-access-92mlk\") pod \"payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk\" (UID: \"7b08cf44-9dc1-47e3-974f-ee121b15559e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk"
Sep 15 21:26:14.192309 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:14.192177    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b08cf44-9dc1-47e3-974f-ee121b15559e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk\" (UID: \"7b08cf44-9dc1-47e3-974f-ee121b15559e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk"
Sep 15 21:26:14.192863 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:14.192844    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b08cf44-9dc1-47e3-974f-ee121b15559e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk\" (UID: \"7b08cf44-9dc1-47e3-974f-ee121b15559e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk"
Sep 15 21:26:14.201058 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:14.201032    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92mlk\" (UniqueName: \"kubernetes.io/projected/7b08cf44-9dc1-47e3-974f-ee121b15559e-kube-api-access-92mlk\") pod \"payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk\" (UID: \"7b08cf44-9dc1-47e3-974f-ee121b15559e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk"
Sep 15 21:26:14.288974 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:14.288936    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk"
Sep 15 21:26:14.577403 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:14.577370    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk"]
Sep 15 21:26:14.579105 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:26:14.579076    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b08cf44_9dc1_47e3_974f_ee121b15559e.slice/crio-7e5944206bce14ead4ec489e4d3213511114d1dd86d67a89425239c6c70944dc WatchSource:0}: Error finding container 7e5944206bce14ead4ec489e4d3213511114d1dd86d67a89425239c6c70944dc: Status 404 returned error can't find the container with id 7e5944206bce14ead4ec489e4d3213511114d1dd86d67a89425239c6c70944dc
Sep 15 21:26:14.764783 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:14.764750    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk" event={"ID":"7b08cf44-9dc1-47e3-974f-ee121b15559e","Type":"ContainerStarted","Data":"543b5090fffc37814d3e0901513d8fffd7311f4783689028b04d36eedface661"}
Sep 15 21:26:14.764972 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:14.764807    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk" event={"ID":"7b08cf44-9dc1-47e3-974f-ee121b15559e","Type":"ContainerStarted","Data":"7e5944206bce14ead4ec489e4d3213511114d1dd86d67a89425239c6c70944dc"}
Sep 15 21:26:14.764972 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:14.764839    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk"
Sep 15 21:26:14.766563 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:14.766531    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5" event={"ID":"f4f1ff0a-c012-494a-8982-0ccefe32e543","Type":"ContainerStarted","Data":"63be8466537be54f53d8d62f583ba2147700b7cbe6cabbbc0335c9aa66b2c153"}
Sep 15 21:26:14.766676 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:14.766587    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5" event={"ID":"f4f1ff0a-c012-494a-8982-0ccefe32e543","Type":"ContainerStarted","Data":"2d117aa7a41602ea2c6718580c2317e597b83c6db7c801fb2d73cc345bce664e"}
Sep 15 21:26:14.766676 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:14.766610    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"
Sep 15 21:26:14.800151 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:14.800099    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk" podStartSLOduration=1.800082966 podStartE2EDuration="1.800082966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:26:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:26:14.781083903 +0000 UTC m=+1711.323246539" watchObservedRunningTime="2026-09-15 21:26:14.800082966 +0000 UTC m=+1711.342245601"
Sep 15 21:26:14.800539 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:14.800512    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5" podStartSLOduration=1.2578877849999999 podStartE2EDuration="1.800479831s" totalImagesPullingTime="542.592046ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:26:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:26:14.799521245 +0000 UTC m=+1711.341683899" watchObservedRunningTime="2026-09-15 21:26:14.800479831 +0000 UTC m=+1711.342642466"
Sep 15 21:26:20.777024 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:20.776993    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"
Sep 15 21:26:25.774510 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:25.774460    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk"
Sep 15 21:26:35.509032 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:35.508990    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:36.455125 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:36.455087    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:36.553417 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:36.553377    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:37.032144 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:37.032109    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:37.208772 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:37.208731    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:37.488235 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:37.488196    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:37.632322 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:37.632289    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:38.294502 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:38.294455    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:39.685060 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:39.685015    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:40.529343 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:40.529306    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:40.691081 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:40.691044    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:40.930365 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:40.930318    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:41.908760 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:41.908725    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:42.780257 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:42.780222    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:43.776474 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:43.776430    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:43.871742 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:43.871704    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:43.976251 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:43.976118    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:44.319453 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:44.319419    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:26:44.418766 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:26:44.418730    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:27:06.050003 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:06.049963    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:27:06.851670 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:06.851627    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:27:07.679996 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:07.679956    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:27:08.408970 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:08.408927    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:27:09.276921 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:09.276883    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:27:10.456056 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:10.456019    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:27:11.583942 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:11.583898    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:27:20.113265 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.113228    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"]
Sep 15 21:27:20.116962 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.116941    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:20.120181 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.120155    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-b8f396\"/\"openshift-service-ca.crt\""
Sep 15 21:27:20.121310 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.121292    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-b8f396\"/\"kube-root-ca.crt\""
Sep 15 21:27:20.121441 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.121322    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-b8f396\"/\"ipp-route-b8f396-kserve-self-signed-certs\""
Sep 15 21:27:20.121637 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.121622    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-b8f396\"/\"default-dockercfg-jmt2f\""
Sep 15 21:27:20.133136 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.133111    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"]
Sep 15 21:27:20.164951 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.164921    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:27:20.220111 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.220063    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-dshm\") pod \"ipp-route-b8f396-kserve-7d5c64d76c-vtwcf\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") " pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:20.220111 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.220121    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-model-cache\") pod \"ipp-route-b8f396-kserve-7d5c64d76c-vtwcf\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") " pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:20.220367 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.220164    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f25b093-c7b4-4a70-b2b8-b87fa921460b-tls-certs\") pod \"ipp-route-b8f396-kserve-7d5c64d76c-vtwcf\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") " pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:20.220367 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.220180    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-tmp-dir\") pod \"ipp-route-b8f396-kserve-7d5c64d76c-vtwcf\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") " pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:20.220367 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.220201    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh5f8\" (UniqueName: \"kubernetes.io/projected/3f25b093-c7b4-4a70-b2b8-b87fa921460b-kube-api-access-zh5f8\") pod \"ipp-route-b8f396-kserve-7d5c64d76c-vtwcf\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") " pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:20.220367 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.220338    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-home\") pod \"ipp-route-b8f396-kserve-7d5c64d76c-vtwcf\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") " pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:20.321859 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.321809    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-home\") pod \"ipp-route-b8f396-kserve-7d5c64d76c-vtwcf\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") " pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:20.321859 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.321864    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-dshm\") pod \"ipp-route-b8f396-kserve-7d5c64d76c-vtwcf\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") " pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:20.322126 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.321890    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-model-cache\") pod \"ipp-route-b8f396-kserve-7d5c64d76c-vtwcf\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") " pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:20.322126 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.321925    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f25b093-c7b4-4a70-b2b8-b87fa921460b-tls-certs\") pod \"ipp-route-b8f396-kserve-7d5c64d76c-vtwcf\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") " pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:20.322126 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.321942    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-tmp-dir\") pod \"ipp-route-b8f396-kserve-7d5c64d76c-vtwcf\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") " pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:20.322126 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.321962    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh5f8\" (UniqueName: \"kubernetes.io/projected/3f25b093-c7b4-4a70-b2b8-b87fa921460b-kube-api-access-zh5f8\") pod \"ipp-route-b8f396-kserve-7d5c64d76c-vtwcf\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") " pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:20.322339 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.322299    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-home\") pod \"ipp-route-b8f396-kserve-7d5c64d76c-vtwcf\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") " pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:20.322382 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.322357    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-model-cache\") pod \"ipp-route-b8f396-kserve-7d5c64d76c-vtwcf\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") " pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:20.322417 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.322393    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-tmp-dir\") pod \"ipp-route-b8f396-kserve-7d5c64d76c-vtwcf\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") " pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:20.324353 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.324335    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-dshm\") pod \"ipp-route-b8f396-kserve-7d5c64d76c-vtwcf\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") " pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:20.324715 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.324700    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f25b093-c7b4-4a70-b2b8-b87fa921460b-tls-certs\") pod \"ipp-route-b8f396-kserve-7d5c64d76c-vtwcf\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") " pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:20.330617 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.330593    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh5f8\" (UniqueName: \"kubernetes.io/projected/3f25b093-c7b4-4a70-b2b8-b87fa921460b-kube-api-access-zh5f8\") pod \"ipp-route-b8f396-kserve-7d5c64d76c-vtwcf\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") " pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:20.427106 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.427003    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:20.711335 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:20.711254    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"]
Sep 15 21:27:20.714685 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:27:20.714643    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f25b093_c7b4_4a70_b2b8_b87fa921460b.slice/crio-9ff3984c5aa047f16aa7cc3e77a54cf50baeaf1fa12f9ad50df76203ebc09cfb WatchSource:0}: Error finding container 9ff3984c5aa047f16aa7cc3e77a54cf50baeaf1fa12f9ad50df76203ebc09cfb: Status 404 returned error can't find the container with id 9ff3984c5aa047f16aa7cc3e77a54cf50baeaf1fa12f9ad50df76203ebc09cfb
Sep 15 21:27:21.107789 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:21.107750    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf" event={"ID":"3f25b093-c7b4-4a70-b2b8-b87fa921460b","Type":"ContainerStarted","Data":"1b3f77803866b8bed2c98f561fce189681a44082233ac5ae3029ffa4dc548eb3"}
Sep 15 21:27:21.107978 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:21.107800    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf" event={"ID":"3f25b093-c7b4-4a70-b2b8-b87fa921460b","Type":"ContainerStarted","Data":"9ff3984c5aa047f16aa7cc3e77a54cf50baeaf1fa12f9ad50df76203ebc09cfb"}
Sep 15 21:27:21.126409 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:21.126357    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf" podStartSLOduration=1.126340037 podStartE2EDuration="1.126340037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:27:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:27:21.125194737 +0000 UTC m=+1777.667357371" watchObservedRunningTime="2026-09-15 21:27:21.126340037 +0000 UTC m=+1777.668502671"
Sep 15 21:27:30.427180 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:30.427129    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:30.427713 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:30.427202    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:30.439963 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:30.439925    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:31.169368 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:31.169339    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:27:44.048518 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:44.048472    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-7t46m_c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf/console-operator/1.log"
Sep 15 21:27:44.051782 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:44.051757    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4c7k_fc205950-caf0-4c47-84ce-a8fc54a1afb2/ovn-acl-logging/0.log"
Sep 15 21:27:44.056527 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:44.056483    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-7t46m_c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf/console-operator/1.log"
Sep 15 21:27:44.060063 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:44.060042    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4c7k_fc205950-caf0-4c47-84ce-a8fc54a1afb2/ovn-acl-logging/0.log"
Sep 15 21:27:46.202129 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:46.202102    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk_7b08cf44-9dc1-47e3-974f-ee121b15559e/payload-processing/0.log"
Sep 15 21:27:46.689846 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:46.689818    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:27:48.592171 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:48.592130    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:27:49.051862 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:49.051827    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:27:49.748170 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:49.748137    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:27:49.841033 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:49.840971    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:27:49.990024 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:49.989988    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:27:50.985233 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:50.985193    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:27:51.681819 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:51.681773    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:27:52.398780 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:52.398740    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:27:53.264465 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:53.264426    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"]
Sep 15 21:27:53.269732 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:53.269703    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"
Sep 15 21:27:53.269959 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:53.269926    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"]
Sep 15 21:27:53.272963 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:53.272940    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-9d9247-serving-cert\""
Sep 15 21:27:53.294819 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:53.294775    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:27:53.337540 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:53.337450    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx\" (UID: \"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"
Sep 15 21:27:53.337540 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:53.337524    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx\" (UID: \"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"
Sep 15 21:27:53.337798 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:53.337626    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2csz\" (UniqueName: \"kubernetes.io/projected/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-kube-api-access-j2csz\") pod \"maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx\" (UID: \"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"
Sep 15 21:27:53.337798 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:53.337659    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx\" (UID: \"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"
Sep 15 21:27:53.395732 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:53.395704    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:27:53.438862 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:53.438828    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx\" (UID: \"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"
Sep 15 21:27:53.439293 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:53.438876    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx\" (UID: \"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"
Sep 15 21:27:53.439293 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:53.439039    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2csz\" (UniqueName: \"kubernetes.io/projected/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-kube-api-access-j2csz\") pod \"maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx\" (UID: \"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"
Sep 15 21:27:53.439293 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:53.439076    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx\" (UID: \"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"
Sep 15 21:27:53.440049 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:53.440020    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx\" (UID: \"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"
Sep 15 21:27:53.441676 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:53.441652    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx\" (UID: \"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"
Sep 15 21:27:53.441791 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:53.441680    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx\" (UID: \"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"
Sep 15 21:27:53.447529 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:53.447505    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2csz\" (UniqueName: \"kubernetes.io/projected/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-kube-api-access-j2csz\") pod \"maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx\" (UID: \"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"
Sep 15 21:27:53.459082 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:53.459040    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:27:53.584076 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:53.583984    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"
Sep 15 21:27:53.793868 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:53.793825    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:27:53.860059 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:53.860029    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"]
Sep 15 21:27:53.862002 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:27:53.861965    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9aae6f8_ebc1_46a0_b15e_8e7e7160b1f8.slice/crio-ef4e0e07f5a37c2da4cdb64b034d98e92d200d4da0effad8eb05b7a24232c74d WatchSource:0}: Error finding container ef4e0e07f5a37c2da4cdb64b034d98e92d200d4da0effad8eb05b7a24232c74d: Status 404 returned error can't find the container with id ef4e0e07f5a37c2da4cdb64b034d98e92d200d4da0effad8eb05b7a24232c74d
Sep 15 21:27:53.863444 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:53.863412    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:27:54.288589 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:54.288559    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx" event={"ID":"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8","Type":"ContainerStarted","Data":"ef4e0e07f5a37c2da4cdb64b034d98e92d200d4da0effad8eb05b7a24232c74d"}
Sep 15 21:27:55.296303 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:55.296265    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx" event={"ID":"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8","Type":"ContainerStarted","Data":"fc89bcbe139a41cf401cf0740129154ef6bdbff36f8b0ad9e2157dd66dfffe86"}
Sep 15 21:27:55.296829 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:55.296332    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"
Sep 15 21:27:59.334987 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:27:59.334954    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:01.307316 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:01.307277    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"
Sep 15 21:28:01.327927 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:01.327747    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx" podStartSLOduration=7.925533741 podStartE2EDuration="8.327730774s" totalImagesPullingTime="402.197033ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:27:55.317538046 +0000 UTC m=+1811.859700692" watchObservedRunningTime="2026-09-15 21:28:01.327730774 +0000 UTC m=+1817.869893410"
Sep 15 21:28:15.395723 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:15.395680    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:16.153783 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:16.153731    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:16.998050 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:16.998010    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:17.990801 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:17.990767    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:18.988128 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:18.988082    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:19.732636 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:19.732593    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:19.832613 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:19.832572    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:21.488279 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:21.488241    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:22.221672 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:22.221640    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:22.949955 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:22.949916    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:23.709143 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:23.709105    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:24.180516 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:24.180456    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:24.249094 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:24.249054    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:24.467236 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:24.467146    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:26.006230 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:26.006196    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:26.263341 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:26.263254    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:31.620066 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:31.620031    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"]
Sep 15 21:28:31.620484 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:31.620268    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx" podUID="d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8" containerName="maas-api" containerID="cri-o://fc89bcbe139a41cf401cf0740129154ef6bdbff36f8b0ad9e2157dd66dfffe86" gracePeriod=30
Sep 15 21:28:31.637626 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:31.637410    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:31.709981 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:31.709915    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:31.772617 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:31.772583    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:31.845728 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:31.845646    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:31.886765 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:31.886737    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"
Sep 15 21:28:31.914649 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:31.914615    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:31.916766 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:31.916617    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-metrics-certs\") pod \"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8\" (UID: \"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8\") "
Sep 15 21:28:31.916938 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:31.916833    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2csz\" (UniqueName: \"kubernetes.io/projected/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-kube-api-access-j2csz\") pod \"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8\" (UID: \"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8\") "
Sep 15 21:28:31.916938 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:31.916911    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-maas-api-tls\") pod \"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8\" (UID: \"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8\") "
Sep 15 21:28:31.917082 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:31.916943    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-openshift-service-ca\") pod \"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8\" (UID: \"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8\") "
Sep 15 21:28:31.917699 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:31.917668    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8" (UID: "d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:28:31.919527 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:31.919467    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8" (UID: "d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:28:31.919716 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:31.919696    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8" (UID: "d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:28:31.919844 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:31.919825    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-kube-api-access-j2csz" (OuterVolumeSpecName: "kube-api-access-j2csz") pod "d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8" (UID: "d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8"). InnerVolumeSpecName "kube-api-access-j2csz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:28:32.018200 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:32.018102    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2csz\" (UniqueName: \"kubernetes.io/projected/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-kube-api-access-j2csz\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:28:32.018200 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:32.018136    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-maas-api-tls\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:28:32.018200 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:32.018149    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-openshift-service-ca\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:28:32.018200 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:32.018162    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8-metrics-certs\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:28:32.221139 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:32.221023    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:32.320280 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:32.320240    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:32.495675 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:32.495594    2564 generic.go:356] "Generic (PLEG): container finished" podID="d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8" containerID="fc89bcbe139a41cf401cf0740129154ef6bdbff36f8b0ad9e2157dd66dfffe86" exitCode=0
Sep 15 21:28:32.495675 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:32.495658    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"
Sep 15 21:28:32.495868 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:32.495679    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx" event={"ID":"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8","Type":"ContainerDied","Data":"fc89bcbe139a41cf401cf0740129154ef6bdbff36f8b0ad9e2157dd66dfffe86"}
Sep 15 21:28:32.495868 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:32.495759    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx" event={"ID":"d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8","Type":"ContainerDied","Data":"ef4e0e07f5a37c2da4cdb64b034d98e92d200d4da0effad8eb05b7a24232c74d"}
Sep 15 21:28:32.495868 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:32.495781    2564 scope.go:117] "RemoveContainer" containerID="fc89bcbe139a41cf401cf0740129154ef6bdbff36f8b0ad9e2157dd66dfffe86"
Sep 15 21:28:32.505156 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:32.505135    2564 scope.go:117] "RemoveContainer" containerID="fc89bcbe139a41cf401cf0740129154ef6bdbff36f8b0ad9e2157dd66dfffe86"
Sep 15 21:28:32.505477 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:28:32.505449    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc89bcbe139a41cf401cf0740129154ef6bdbff36f8b0ad9e2157dd66dfffe86\": container with ID starting with fc89bcbe139a41cf401cf0740129154ef6bdbff36f8b0ad9e2157dd66dfffe86 not found: ID does not exist" containerID="fc89bcbe139a41cf401cf0740129154ef6bdbff36f8b0ad9e2157dd66dfffe86"
Sep 15 21:28:32.505538 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:32.505510    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc89bcbe139a41cf401cf0740129154ef6bdbff36f8b0ad9e2157dd66dfffe86"} err="failed to get container status \"fc89bcbe139a41cf401cf0740129154ef6bdbff36f8b0ad9e2157dd66dfffe86\": rpc error: code = NotFound desc = could not find container \"fc89bcbe139a41cf401cf0740129154ef6bdbff36f8b0ad9e2157dd66dfffe86\": container with ID starting with fc89bcbe139a41cf401cf0740129154ef6bdbff36f8b0ad9e2157dd66dfffe86 not found: ID does not exist"
Sep 15 21:28:32.514701 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:32.514672    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"]
Sep 15 21:28:32.519150 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:32.519126    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9d9247-589856b5c4-6hgjx"]
Sep 15 21:28:33.030714 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:33.030683    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:33.973353 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:33.973317    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8" path="/var/lib/kubelet/pods/d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8/volumes"
Sep 15 21:28:33.976070 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:33.976035    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:34.698789 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:34.698754    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:35.428552 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:35.428513    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:38.008965 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:38.008932    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:38.108999 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:38.108959    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:38.262384 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:38.262287    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:41.886566 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:41.886528    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:42.693263 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:42.693223    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:43.681019 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:43.680981    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:45.188006 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:45.187966    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:45.288834 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:45.288795    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:45.920417 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:45.920384    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:57.376770 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.376733    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"]
Sep 15 21:28:57.377591 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.377543    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5" podUID="f4f1ff0a-c012-494a-8982-0ccefe32e543" containerName="maas-api" containerID="cri-o://63be8466537be54f53d8d62f583ba2147700b7cbe6cabbbc0335c9aa66b2c153" gracePeriod=30
Sep 15 21:28:57.409818 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.409784    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:57.434828 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.434791    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk"]
Sep 15 21:28:57.435080 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.435049    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk" podUID="7b08cf44-9dc1-47e3-974f-ee121b15559e" containerName="payload-processing" containerID="cri-o://543b5090fffc37814d3e0901513d8fffd7311f4783689028b04d36eedface661" gracePeriod=30
Sep 15 21:28:57.509975 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.509931    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:57.610832 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.610752    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:57.629172 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.628796    2564 generic.go:356] "Generic (PLEG): container finished" podID="7b08cf44-9dc1-47e3-974f-ee121b15559e" containerID="543b5090fffc37814d3e0901513d8fffd7311f4783689028b04d36eedface661" exitCode=0
Sep 15 21:28:57.629172 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.628912    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk" event={"ID":"7b08cf44-9dc1-47e3-974f-ee121b15559e","Type":"ContainerDied","Data":"543b5090fffc37814d3e0901513d8fffd7311f4783689028b04d36eedface661"}
Sep 15 21:28:57.631271 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.631117    2564 generic.go:356] "Generic (PLEG): container finished" podID="f4f1ff0a-c012-494a-8982-0ccefe32e543" containerID="63be8466537be54f53d8d62f583ba2147700b7cbe6cabbbc0335c9aa66b2c153" exitCode=0
Sep 15 21:28:57.631271 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.631194    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5" event={"ID":"f4f1ff0a-c012-494a-8982-0ccefe32e543","Type":"ContainerDied","Data":"63be8466537be54f53d8d62f583ba2147700b7cbe6cabbbc0335c9aa66b2c153"}
Sep 15 21:28:57.699575 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.699547    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"
Sep 15 21:28:57.712540 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.712484    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:57.728528 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.728472    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk"
Sep 15 21:28:57.870739 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.870700    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f1ff0a-c012-494a-8982-0ccefe32e543-metrics-certs\") pod \"f4f1ff0a-c012-494a-8982-0ccefe32e543\" (UID: \"f4f1ff0a-c012-494a-8982-0ccefe32e543\") "
Sep 15 21:28:57.870945 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.870757    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b08cf44-9dc1-47e3-974f-ee121b15559e-plugins-config-volume\") pod \"7b08cf44-9dc1-47e3-974f-ee121b15559e\" (UID: \"7b08cf44-9dc1-47e3-974f-ee121b15559e\") "
Sep 15 21:28:57.870945 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.870797    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92mlk\" (UniqueName: \"kubernetes.io/projected/7b08cf44-9dc1-47e3-974f-ee121b15559e-kube-api-access-92mlk\") pod \"7b08cf44-9dc1-47e3-974f-ee121b15559e\" (UID: \"7b08cf44-9dc1-47e3-974f-ee121b15559e\") "
Sep 15 21:28:57.870945 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.870831    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m7bzf\" (UniqueName: \"kubernetes.io/projected/f4f1ff0a-c012-494a-8982-0ccefe32e543-kube-api-access-m7bzf\") pod \"f4f1ff0a-c012-494a-8982-0ccefe32e543\" (UID: \"f4f1ff0a-c012-494a-8982-0ccefe32e543\") "
Sep 15 21:28:57.870945 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.870850    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4f1ff0a-c012-494a-8982-0ccefe32e543-maas-api-tls\") pod \"f4f1ff0a-c012-494a-8982-0ccefe32e543\" (UID: \"f4f1ff0a-c012-494a-8982-0ccefe32e543\") "
Sep 15 21:28:57.870945 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.870884    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4f1ff0a-c012-494a-8982-0ccefe32e543-openshift-service-ca\") pod \"f4f1ff0a-c012-494a-8982-0ccefe32e543\" (UID: \"f4f1ff0a-c012-494a-8982-0ccefe32e543\") "
Sep 15 21:28:57.871271 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.871224    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b08cf44-9dc1-47e3-974f-ee121b15559e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7b08cf44-9dc1-47e3-974f-ee121b15559e" (UID: "7b08cf44-9dc1-47e3-974f-ee121b15559e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:28:57.871331 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.871295    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4f1ff0a-c012-494a-8982-0ccefe32e543-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f4f1ff0a-c012-494a-8982-0ccefe32e543" (UID: "f4f1ff0a-c012-494a-8982-0ccefe32e543"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:28:57.873210 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.873185    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4f1ff0a-c012-494a-8982-0ccefe32e543-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f4f1ff0a-c012-494a-8982-0ccefe32e543" (UID: "f4f1ff0a-c012-494a-8982-0ccefe32e543"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:28:57.873210 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.873199    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b08cf44-9dc1-47e3-974f-ee121b15559e-kube-api-access-92mlk" (OuterVolumeSpecName: "kube-api-access-92mlk") pod "7b08cf44-9dc1-47e3-974f-ee121b15559e" (UID: "7b08cf44-9dc1-47e3-974f-ee121b15559e"). InnerVolumeSpecName "kube-api-access-92mlk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:28:57.873616 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.873587    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4f1ff0a-c012-494a-8982-0ccefe32e543-kube-api-access-m7bzf" (OuterVolumeSpecName: "kube-api-access-m7bzf") pod "f4f1ff0a-c012-494a-8982-0ccefe32e543" (UID: "f4f1ff0a-c012-494a-8982-0ccefe32e543"). InnerVolumeSpecName "kube-api-access-m7bzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:28:57.873616 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.873598    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4f1ff0a-c012-494a-8982-0ccefe32e543-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f4f1ff0a-c012-494a-8982-0ccefe32e543" (UID: "f4f1ff0a-c012-494a-8982-0ccefe32e543"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:28:57.972453 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.972364    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7bzf\" (UniqueName: \"kubernetes.io/projected/f4f1ff0a-c012-494a-8982-0ccefe32e543-kube-api-access-m7bzf\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:28:57.972453 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.972402    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4f1ff0a-c012-494a-8982-0ccefe32e543-maas-api-tls\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:28:57.972453 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.972418    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4f1ff0a-c012-494a-8982-0ccefe32e543-openshift-service-ca\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:28:57.972453 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.972433    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f1ff0a-c012-494a-8982-0ccefe32e543-metrics-certs\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:28:57.972453 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.972448    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b08cf44-9dc1-47e3-974f-ee121b15559e-plugins-config-volume\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:28:57.972745 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.972462    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92mlk\" (UniqueName: \"kubernetes.io/projected/7b08cf44-9dc1-47e3-974f-ee121b15559e-kube-api-access-92mlk\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:28:57.999017 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:57.998964    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:58.184668 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:58.184624    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:58.638260 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:58.638230    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"
Sep 15 21:28:58.638736 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:58.638268    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5" event={"ID":"f4f1ff0a-c012-494a-8982-0ccefe32e543","Type":"ContainerDied","Data":"2d117aa7a41602ea2c6718580c2317e597b83c6db7c801fb2d73cc345bce664e"}
Sep 15 21:28:58.638736 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:58.638324    2564 scope.go:117] "RemoveContainer" containerID="63be8466537be54f53d8d62f583ba2147700b7cbe6cabbbc0335c9aa66b2c153"
Sep 15 21:28:58.640045 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:58.640019    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk" event={"ID":"7b08cf44-9dc1-47e3-974f-ee121b15559e","Type":"ContainerDied","Data":"7e5944206bce14ead4ec489e4d3213511114d1dd86d67a89425239c6c70944dc"}
Sep 15 21:28:58.640235 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:58.640217    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk"
Sep 15 21:28:58.647919 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:58.647904    2564 scope.go:117] "RemoveContainer" containerID="543b5090fffc37814d3e0901513d8fffd7311f4783689028b04d36eedface661"
Sep 15 21:28:58.668266 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:58.668229    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk"]
Sep 15 21:28:58.675115 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:58.675083    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-b8f396-7ccdfbbc9c-tbgxk"]
Sep 15 21:28:58.691458 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:58.691420    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"]
Sep 15 21:28:58.694706 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:58.694676    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b8f396-7844f97bf8-6qxz5"]
Sep 15 21:28:59.152677 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:59.152633    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:28:59.972506 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:59.972458    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b08cf44-9dc1-47e3-974f-ee121b15559e" path="/var/lib/kubelet/pods/7b08cf44-9dc1-47e3-974f-ee121b15559e/volumes"
Sep 15 21:28:59.972897 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:28:59.972885    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4f1ff0a-c012-494a-8982-0ccefe32e543" path="/var/lib/kubelet/pods/f4f1ff0a-c012-494a-8982-0ccefe32e543/volumes"
Sep 15 21:29:00.058936 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:00.058900    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:29:01.174411 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:01.174371    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:29:03.062274 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:03.062235    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"]
Sep 15 21:29:03.796836 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:03.796790    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:29:11.085369 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:11.085273    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:29:18.082278 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.082232    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf" podUID="3f25b093-c7b4-4a70-b2b8-b87fa921460b" containerName="main" containerID="cri-o://1b3f77803866b8bed2c98f561fce189681a44082233ac5ae3029ffa4dc548eb3" gracePeriod=45
Sep 15 21:29:18.334530 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.334433    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:29:18.344643 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.344610    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zh5f8\" (UniqueName: \"kubernetes.io/projected/3f25b093-c7b4-4a70-b2b8-b87fa921460b-kube-api-access-zh5f8\") pod \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") "
Sep 15 21:29:18.344829 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.344684    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-dshm\") pod \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") "
Sep 15 21:29:18.344829 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.344710    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-home\") pod \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") "
Sep 15 21:29:18.344829 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.344793    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-model-cache\") pod \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") "
Sep 15 21:29:18.345014 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.344832    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f25b093-c7b4-4a70-b2b8-b87fa921460b-tls-certs\") pod \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") "
Sep 15 21:29:18.345014 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.344857    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-tmp-dir\") pod \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\" (UID: \"3f25b093-c7b4-4a70-b2b8-b87fa921460b\") "
Sep 15 21:29:18.345137 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.345028    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-home" (OuterVolumeSpecName: "home") pod "3f25b093-c7b4-4a70-b2b8-b87fa921460b" (UID: "3f25b093-c7b4-4a70-b2b8-b87fa921460b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:29:18.345246 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.345226    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-home\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:29:18.345307 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.345250    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3f25b093-c7b4-4a70-b2b8-b87fa921460b" (UID: "3f25b093-c7b4-4a70-b2b8-b87fa921460b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:29:18.345307 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.345264    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-model-cache" (OuterVolumeSpecName: "model-cache") pod "3f25b093-c7b4-4a70-b2b8-b87fa921460b" (UID: "3f25b093-c7b4-4a70-b2b8-b87fa921460b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:29:18.347305 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.347267    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-dshm" (OuterVolumeSpecName: "dshm") pod "3f25b093-c7b4-4a70-b2b8-b87fa921460b" (UID: "3f25b093-c7b4-4a70-b2b8-b87fa921460b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:29:18.347421 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.347390    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f25b093-c7b4-4a70-b2b8-b87fa921460b-kube-api-access-zh5f8" (OuterVolumeSpecName: "kube-api-access-zh5f8") pod "3f25b093-c7b4-4a70-b2b8-b87fa921460b" (UID: "3f25b093-c7b4-4a70-b2b8-b87fa921460b"). InnerVolumeSpecName "kube-api-access-zh5f8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:29:18.347470 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.347417    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f25b093-c7b4-4a70-b2b8-b87fa921460b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3f25b093-c7b4-4a70-b2b8-b87fa921460b" (UID: "3f25b093-c7b4-4a70-b2b8-b87fa921460b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:29:18.446353 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.446324    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-dshm\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:29:18.446353 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.446350    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-model-cache\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:29:18.446353 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.446362    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f25b093-c7b4-4a70-b2b8-b87fa921460b-tls-certs\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:29:18.446620 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.446370    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f25b093-c7b4-4a70-b2b8-b87fa921460b-tmp-dir\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:29:18.446620 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.446378    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zh5f8\" (UniqueName: \"kubernetes.io/projected/3f25b093-c7b4-4a70-b2b8-b87fa921460b-kube-api-access-zh5f8\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:29:18.744832 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.744795    2564 generic.go:356] "Generic (PLEG): container finished" podID="3f25b093-c7b4-4a70-b2b8-b87fa921460b" containerID="1b3f77803866b8bed2c98f561fce189681a44082233ac5ae3029ffa4dc548eb3" exitCode=0
Sep 15 21:29:18.745031 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.744841    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"
Sep 15 21:29:18.745031 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.744886    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf" event={"ID":"3f25b093-c7b4-4a70-b2b8-b87fa921460b","Type":"ContainerDied","Data":"1b3f77803866b8bed2c98f561fce189681a44082233ac5ae3029ffa4dc548eb3"}
Sep 15 21:29:18.745031 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.744930    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf" event={"ID":"3f25b093-c7b4-4a70-b2b8-b87fa921460b","Type":"ContainerDied","Data":"9ff3984c5aa047f16aa7cc3e77a54cf50baeaf1fa12f9ad50df76203ebc09cfb"}
Sep 15 21:29:18.745031 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.744948    2564 scope.go:117] "RemoveContainer" containerID="1b3f77803866b8bed2c98f561fce189681a44082233ac5ae3029ffa4dc548eb3"
Sep 15 21:29:18.754252 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.754230    2564 scope.go:117] "RemoveContainer" containerID="1b3f77803866b8bed2c98f561fce189681a44082233ac5ae3029ffa4dc548eb3"
Sep 15 21:29:18.754560 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:29:18.754542    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b3f77803866b8bed2c98f561fce189681a44082233ac5ae3029ffa4dc548eb3\": container with ID starting with 1b3f77803866b8bed2c98f561fce189681a44082233ac5ae3029ffa4dc548eb3 not found: ID does not exist" containerID="1b3f77803866b8bed2c98f561fce189681a44082233ac5ae3029ffa4dc548eb3"
Sep 15 21:29:18.754620 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.754569    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b3f77803866b8bed2c98f561fce189681a44082233ac5ae3029ffa4dc548eb3"} err="failed to get container status \"1b3f77803866b8bed2c98f561fce189681a44082233ac5ae3029ffa4dc548eb3\": rpc error: code = NotFound desc = could not find container \"1b3f77803866b8bed2c98f561fce189681a44082233ac5ae3029ffa4dc548eb3\": container with ID starting with 1b3f77803866b8bed2c98f561fce189681a44082233ac5ae3029ffa4dc548eb3 not found: ID does not exist"
Sep 15 21:29:18.777691 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.777640    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"]
Sep 15 21:29:18.785986 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:18.785956    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-b8f396/ipp-route-b8f396-kserve-7d5c64d76c-vtwcf"]
Sep 15 21:29:19.975344 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:19.974384    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f25b093-c7b4-4a70-b2b8-b87fa921460b" path="/var/lib/kubelet/pods/3f25b093-c7b4-4a70-b2b8-b87fa921460b/volumes"
Sep 15 21:29:40.375533 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:40.375478    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:29:41.144621 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:41.144573    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:29:47.166191 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:47.166149    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:29:53.195956 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:53.195914    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:29:53.270355 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:53.270316    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:29:53.505525 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:53.505416    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:29:54.149697 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:54.149654    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:29:55.140374 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:55.140328    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:29:55.587676 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:55.587592    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:29:55.983741 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:55.983704    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:29:56.754299 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:56.754260    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:29:57.474381 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:29:57.474341    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:00.201998 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:00.201963    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2f34ef-29825130-pr82t"]
Sep 15 21:30:00.202383 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:00.202351    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8" containerName="maas-api"
Sep 15 21:30:00.202383 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:00.202362    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8" containerName="maas-api"
Sep 15 21:30:00.202383 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:00.202377    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b08cf44-9dc1-47e3-974f-ee121b15559e" containerName="payload-processing"
Sep 15 21:30:00.202383 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:00.202383    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b08cf44-9dc1-47e3-974f-ee121b15559e" containerName="payload-processing"
Sep 15 21:30:00.202535 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:00.202396    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f25b093-c7b4-4a70-b2b8-b87fa921460b" containerName="main"
Sep 15 21:30:00.202535 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:00.202401    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f25b093-c7b4-4a70-b2b8-b87fa921460b" containerName="main"
Sep 15 21:30:00.202535 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:00.202409    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4f1ff0a-c012-494a-8982-0ccefe32e543" containerName="maas-api"
Sep 15 21:30:00.202535 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:00.202415    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4f1ff0a-c012-494a-8982-0ccefe32e543" containerName="maas-api"
Sep 15 21:30:00.202535 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:00.202481    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f25b093-c7b4-4a70-b2b8-b87fa921460b" containerName="main"
Sep 15 21:30:00.202535 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:00.202510    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b08cf44-9dc1-47e3-974f-ee121b15559e" containerName="payload-processing"
Sep 15 21:30:00.202535 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:00.202516    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4f1ff0a-c012-494a-8982-0ccefe32e543" containerName="maas-api"
Sep 15 21:30:00.202535 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:00.202522    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9aae6f8-ebc1-46a0-b15e-8e7e7160b1f8" containerName="maas-api"
Sep 15 21:30:00.206216 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:00.206193    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2f34ef-29825130-pr82t"
Sep 15 21:30:00.210028 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:00.210002    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vvf2c\""
Sep 15 21:30:00.216844 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:00.216819    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2f34ef-29825130-pr82t"
Sep 15 21:30:00.239589 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:00.239504    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2f34ef-29825130-pr82t"]
Sep 15 21:30:00.493063 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:00.492975    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2f34ef-29825130-pr82t"]
Sep 15 21:30:00.496413 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:30:00.496381    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2beb8a01_4a23_4adf_9ddd_aa76d218527b.slice/crio-043f892b9be4911aaf6a979acad9255221fc050c06f4cfe6229dbe9bca44a328 WatchSource:0}: Error finding container 043f892b9be4911aaf6a979acad9255221fc050c06f4cfe6229dbe9bca44a328: Status 404 returned error can't find the container with id 043f892b9be4911aaf6a979acad9255221fc050c06f4cfe6229dbe9bca44a328
Sep 15 21:30:00.951252 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:00.951213    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2f34ef-29825130-pr82t" event={"ID":"2beb8a01-4a23-4adf-9ddd-aa76d218527b","Type":"ContainerStarted","Data":"043f892b9be4911aaf6a979acad9255221fc050c06f4cfe6229dbe9bca44a328"}
Sep 15 21:30:03.971411 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:03.971322    2564 generic.go:356] "Generic (PLEG): container finished" podID="2beb8a01-4a23-4adf-9ddd-aa76d218527b" containerID="38a9b241a03649cf41543e9930f3a7a8a39112245e1a670bb7dc22e66bc4293f" exitCode=0
Sep 15 21:30:03.973115 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:03.973092    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2f34ef-29825130-pr82t" event={"ID":"2beb8a01-4a23-4adf-9ddd-aa76d218527b","Type":"ContainerDied","Data":"38a9b241a03649cf41543e9930f3a7a8a39112245e1a670bb7dc22e66bc4293f"}
Sep 15 21:30:04.583348 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:04.583312    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:04.664884 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:04.664843    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:04.765701 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:04.765665    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:04.847947 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:04.847910    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:05.134149 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:05.134119    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2f34ef-29825130-pr82t"
Sep 15 21:30:05.295439 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:05.295403    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:05.482346 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:05.482254    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:05.992562 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:05.992529    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2f34ef-29825130-pr82t" event={"ID":"2beb8a01-4a23-4adf-9ddd-aa76d218527b","Type":"ContainerDied","Data":"043f892b9be4911aaf6a979acad9255221fc050c06f4cfe6229dbe9bca44a328"}
Sep 15 21:30:05.992562 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:05.992550    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2f34ef-29825130-pr82t"
Sep 15 21:30:05.992804 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:05.992579    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="043f892b9be4911aaf6a979acad9255221fc050c06f4cfe6229dbe9bca44a328"
Sep 15 21:30:06.427479 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:06.427438    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:07.420252 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:07.420215    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:11.296121 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:11.296087    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:11.394142 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:11.394103    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:11.560294 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:11.560202    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:12.645367 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:12.645335    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:13.805766 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:13.805726    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:14.476308 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:14.476263    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:22.528011 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:22.527973    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2f34ef-29825130-pr82t"]
Sep 15 21:30:22.530002 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:22.529975    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2f34ef-29825130-pr82t"]
Sep 15 21:30:22.533605 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:22.533552    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:22.563855 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:22.563824    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66"]
Sep 15 21:30:22.564077 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:22.564045    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66" podUID="20a0b584-3f13-44d5-bce5-87e6f46927db" containerName="payload-processing" containerID="cri-o://d3a090ad15b37413c985f16e24ee5c17a49db56d18813e1cd2142eac0d456e90" gracePeriod=30
Sep 15 21:30:22.634348 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:22.634298    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:22.736322 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:22.736293    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:22.835065 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:22.835037    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66"
Sep 15 21:30:22.873471 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:22.873432    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:22.952720 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:22.952681    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20a0b584-3f13-44d5-bce5-87e6f46927db-plugins-config-volume\") pod \"20a0b584-3f13-44d5-bce5-87e6f46927db\" (UID: \"20a0b584-3f13-44d5-bce5-87e6f46927db\") "
Sep 15 21:30:22.952944 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:22.952775    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgtmw\" (UniqueName: \"kubernetes.io/projected/20a0b584-3f13-44d5-bce5-87e6f46927db-kube-api-access-dgtmw\") pod \"20a0b584-3f13-44d5-bce5-87e6f46927db\" (UID: \"20a0b584-3f13-44d5-bce5-87e6f46927db\") "
Sep 15 21:30:22.953154 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:22.953130    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20a0b584-3f13-44d5-bce5-87e6f46927db-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "20a0b584-3f13-44d5-bce5-87e6f46927db" (UID: "20a0b584-3f13-44d5-bce5-87e6f46927db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:30:22.955150 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:22.955125    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20a0b584-3f13-44d5-bce5-87e6f46927db-kube-api-access-dgtmw" (OuterVolumeSpecName: "kube-api-access-dgtmw") pod "20a0b584-3f13-44d5-bce5-87e6f46927db" (UID: "20a0b584-3f13-44d5-bce5-87e6f46927db"). InnerVolumeSpecName "kube-api-access-dgtmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:30:23.054254 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:23.054206    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20a0b584-3f13-44d5-bce5-87e6f46927db-plugins-config-volume\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:30:23.054254 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:23.054250    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgtmw\" (UniqueName: \"kubernetes.io/projected/20a0b584-3f13-44d5-bce5-87e6f46927db-kube-api-access-dgtmw\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:30:23.092132 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:23.092032    2564 generic.go:356] "Generic (PLEG): container finished" podID="20a0b584-3f13-44d5-bce5-87e6f46927db" containerID="d3a090ad15b37413c985f16e24ee5c17a49db56d18813e1cd2142eac0d456e90" exitCode=0
Sep 15 21:30:23.092132 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:23.092121    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66"
Sep 15 21:30:23.092330 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:23.092119    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66" event={"ID":"20a0b584-3f13-44d5-bce5-87e6f46927db","Type":"ContainerDied","Data":"d3a090ad15b37413c985f16e24ee5c17a49db56d18813e1cd2142eac0d456e90"}
Sep 15 21:30:23.092330 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:23.092275    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66" event={"ID":"20a0b584-3f13-44d5-bce5-87e6f46927db","Type":"ContainerDied","Data":"f04fdea7f24950962f536ed62e0cffe05d0902c41f4312f632e9340432ae8e47"}
Sep 15 21:30:23.092330 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:23.092299    2564 scope.go:117] "RemoveContainer" containerID="d3a090ad15b37413c985f16e24ee5c17a49db56d18813e1cd2142eac0d456e90"
Sep 15 21:30:23.101836 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:23.101808    2564 scope.go:117] "RemoveContainer" containerID="d3a090ad15b37413c985f16e24ee5c17a49db56d18813e1cd2142eac0d456e90"
Sep 15 21:30:23.102186 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:30:23.102164    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3a090ad15b37413c985f16e24ee5c17a49db56d18813e1cd2142eac0d456e90\": container with ID starting with d3a090ad15b37413c985f16e24ee5c17a49db56d18813e1cd2142eac0d456e90 not found: ID does not exist" containerID="d3a090ad15b37413c985f16e24ee5c17a49db56d18813e1cd2142eac0d456e90"
Sep 15 21:30:23.102245 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:23.102196    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3a090ad15b37413c985f16e24ee5c17a49db56d18813e1cd2142eac0d456e90"} err="failed to get container status \"d3a090ad15b37413c985f16e24ee5c17a49db56d18813e1cd2142eac0d456e90\": rpc error: code = NotFound desc = could not find container \"d3a090ad15b37413c985f16e24ee5c17a49db56d18813e1cd2142eac0d456e90\": container with ID starting with d3a090ad15b37413c985f16e24ee5c17a49db56d18813e1cd2142eac0d456e90 not found: ID does not exist"
Sep 15 21:30:23.115340 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:23.115299    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66"]
Sep 15 21:30:23.117689 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:23.117655    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:23.117963 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:23.117943    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2f34ef-65b4f5848c-4gp66"]
Sep 15 21:30:23.218393 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:23.218354    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:23.973430 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:23.973398    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20a0b584-3f13-44d5-bce5-87e6f46927db" path="/var/lib/kubelet/pods/20a0b584-3f13-44d5-bce5-87e6f46927db/volumes"
Sep 15 21:30:23.973854 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:23.973757    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2beb8a01-4a23-4adf-9ddd-aa76d218527b" path="/var/lib/kubelet/pods/2beb8a01-4a23-4adf-9ddd-aa76d218527b/volumes"
Sep 15 21:30:24.005947 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:24.005915    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:28.977236 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:28.977201    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:29.077313 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:29.077280    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:29.256429 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:29.256327    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:30.376566 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:30.376524    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:31.287815 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:31.287778    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:40.155820 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:40.155736    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:40.193993 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:40.193955    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb"]
Sep 15 21:30:40.194284 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:40.194237    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb" podUID="deb8a238-e92b-435e-964b-fee577c19776" containerName="payload-processing" containerID="cri-o://345256b9bc545bb0d3ca33431da2355fade26b4ac840f5488ff1f77cd1823a80" gracePeriod=30
Sep 15 21:30:40.253267 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:40.253230    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:40.353774 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:40.353728    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:40.453370 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:40.453343    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:40.460705 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:40.460682    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb"
Sep 15 21:30:40.515476 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:40.515415    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/deb8a238-e92b-435e-964b-fee577c19776-plugins-config-volume\") pod \"deb8a238-e92b-435e-964b-fee577c19776\" (UID: \"deb8a238-e92b-435e-964b-fee577c19776\") "
Sep 15 21:30:40.515669 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:40.515556    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpsrg\" (UniqueName: \"kubernetes.io/projected/deb8a238-e92b-435e-964b-fee577c19776-kube-api-access-gpsrg\") pod \"deb8a238-e92b-435e-964b-fee577c19776\" (UID: \"deb8a238-e92b-435e-964b-fee577c19776\") "
Sep 15 21:30:40.515847 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:40.515825    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/deb8a238-e92b-435e-964b-fee577c19776-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "deb8a238-e92b-435e-964b-fee577c19776" (UID: "deb8a238-e92b-435e-964b-fee577c19776"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:30:40.517987 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:40.517960    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/deb8a238-e92b-435e-964b-fee577c19776-kube-api-access-gpsrg" (OuterVolumeSpecName: "kube-api-access-gpsrg") pod "deb8a238-e92b-435e-964b-fee577c19776" (UID: "deb8a238-e92b-435e-964b-fee577c19776"). InnerVolumeSpecName "kube-api-access-gpsrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:30:40.617044 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:40.617004    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpsrg\" (UniqueName: \"kubernetes.io/projected/deb8a238-e92b-435e-964b-fee577c19776-kube-api-access-gpsrg\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:30:40.617044 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:40.617034    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/deb8a238-e92b-435e-964b-fee577c19776-plugins-config-volume\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:30:40.778098 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:40.778006    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:40.876330 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:40.876297    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:41.183104 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:41.183071    2564 generic.go:356] "Generic (PLEG): container finished" podID="deb8a238-e92b-435e-964b-fee577c19776" containerID="345256b9bc545bb0d3ca33431da2355fade26b4ac840f5488ff1f77cd1823a80" exitCode=0
Sep 15 21:30:41.183590 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:41.183112    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb"
Sep 15 21:30:41.183590 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:41.183165    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb" event={"ID":"deb8a238-e92b-435e-964b-fee577c19776","Type":"ContainerDied","Data":"345256b9bc545bb0d3ca33431da2355fade26b4ac840f5488ff1f77cd1823a80"}
Sep 15 21:30:41.183590 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:41.183212    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb" event={"ID":"deb8a238-e92b-435e-964b-fee577c19776","Type":"ContainerDied","Data":"41304bec3d48404e18a855bf16acad3095f241f7ce1f9f526f3a58ae559afcb6"}
Sep 15 21:30:41.183590 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:41.183236    2564 scope.go:117] "RemoveContainer" containerID="345256b9bc545bb0d3ca33431da2355fade26b4ac840f5488ff1f77cd1823a80"
Sep 15 21:30:41.193064 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:41.193046    2564 scope.go:117] "RemoveContainer" containerID="345256b9bc545bb0d3ca33431da2355fade26b4ac840f5488ff1f77cd1823a80"
Sep 15 21:30:41.193375 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:30:41.193350    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"345256b9bc545bb0d3ca33431da2355fade26b4ac840f5488ff1f77cd1823a80\": container with ID starting with 345256b9bc545bb0d3ca33431da2355fade26b4ac840f5488ff1f77cd1823a80 not found: ID does not exist" containerID="345256b9bc545bb0d3ca33431da2355fade26b4ac840f5488ff1f77cd1823a80"
Sep 15 21:30:41.193476 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:41.193380    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"345256b9bc545bb0d3ca33431da2355fade26b4ac840f5488ff1f77cd1823a80"} err="failed to get container status \"345256b9bc545bb0d3ca33431da2355fade26b4ac840f5488ff1f77cd1823a80\": rpc error: code = NotFound desc = could not find container \"345256b9bc545bb0d3ca33431da2355fade26b4ac840f5488ff1f77cd1823a80\": container with ID starting with 345256b9bc545bb0d3ca33431da2355fade26b4ac840f5488ff1f77cd1823a80 not found: ID does not exist"
Sep 15 21:30:41.219837 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:41.219809    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb"]
Sep 15 21:30:41.237854 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:41.237827    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-41ffdf-f94d6cc7f-cj8pb"]
Sep 15 21:30:41.972505 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:41.972462    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="deb8a238-e92b-435e-964b-fee577c19776" path="/var/lib/kubelet/pods/deb8a238-e92b-435e-964b-fee577c19776/volumes"
Sep 15 21:30:46.841970 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:46.841924    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:46.942559 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:46.942517    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:47.276183 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:47.276136    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:47.403764 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:47.403728    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:47.574886 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:47.574783    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:30:50.085202 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:50.085163    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b6486456d-h4hnf"]
Sep 15 21:30:50.085782 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:50.085454    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7b6486456d-h4hnf" podUID="a8b7760f-2f30-4bd8-b87e-afeb2cd908b9" containerName="manager" containerID="cri-o://514c068f4887036125ca3d13f50e6392b5bbcf27363e9f5698235db63dc7cf88" gracePeriod=30
Sep 15 21:30:50.541701 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:50.541671    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b6486456d-h4hnf"
Sep 15 21:30:50.604441 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:50.604404    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8b7760f-2f30-4bd8-b87e-afeb2cd908b9-metrics-certs\") pod \"a8b7760f-2f30-4bd8-b87e-afeb2cd908b9\" (UID: \"a8b7760f-2f30-4bd8-b87e-afeb2cd908b9\") "
Sep 15 21:30:50.604619 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:50.604462    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vz8p\" (UniqueName: \"kubernetes.io/projected/a8b7760f-2f30-4bd8-b87e-afeb2cd908b9-kube-api-access-7vz8p\") pod \"a8b7760f-2f30-4bd8-b87e-afeb2cd908b9\" (UID: \"a8b7760f-2f30-4bd8-b87e-afeb2cd908b9\") "
Sep 15 21:30:50.604619 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:50.604595    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a8b7760f-2f30-4bd8-b87e-afeb2cd908b9-webhook-cert\") pod \"a8b7760f-2f30-4bd8-b87e-afeb2cd908b9\" (UID: \"a8b7760f-2f30-4bd8-b87e-afeb2cd908b9\") "
Sep 15 21:30:50.606791 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:50.606753    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8b7760f-2f30-4bd8-b87e-afeb2cd908b9-kube-api-access-7vz8p" (OuterVolumeSpecName: "kube-api-access-7vz8p") pod "a8b7760f-2f30-4bd8-b87e-afeb2cd908b9" (UID: "a8b7760f-2f30-4bd8-b87e-afeb2cd908b9"). InnerVolumeSpecName "kube-api-access-7vz8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:30:50.606903 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:50.606835    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8b7760f-2f30-4bd8-b87e-afeb2cd908b9-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a8b7760f-2f30-4bd8-b87e-afeb2cd908b9" (UID: "a8b7760f-2f30-4bd8-b87e-afeb2cd908b9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:30:50.606903 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:50.606878    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8b7760f-2f30-4bd8-b87e-afeb2cd908b9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a8b7760f-2f30-4bd8-b87e-afeb2cd908b9" (UID: "a8b7760f-2f30-4bd8-b87e-afeb2cd908b9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:30:50.705645 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:50.705551    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a8b7760f-2f30-4bd8-b87e-afeb2cd908b9-webhook-cert\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:30:50.705645 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:50.705588    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8b7760f-2f30-4bd8-b87e-afeb2cd908b9-metrics-certs\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:30:50.705645 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:50.705597    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vz8p\" (UniqueName: \"kubernetes.io/projected/a8b7760f-2f30-4bd8-b87e-afeb2cd908b9-kube-api-access-7vz8p\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:30:51.239346 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:51.239314    2564 generic.go:356] "Generic (PLEG): container finished" podID="a8b7760f-2f30-4bd8-b87e-afeb2cd908b9" containerID="514c068f4887036125ca3d13f50e6392b5bbcf27363e9f5698235db63dc7cf88" exitCode=0
Sep 15 21:30:51.239819 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:51.239353    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b6486456d-h4hnf" event={"ID":"a8b7760f-2f30-4bd8-b87e-afeb2cd908b9","Type":"ContainerDied","Data":"514c068f4887036125ca3d13f50e6392b5bbcf27363e9f5698235db63dc7cf88"}
Sep 15 21:30:51.239819 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:51.239361    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b6486456d-h4hnf"
Sep 15 21:30:51.239819 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:51.239399    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b6486456d-h4hnf" event={"ID":"a8b7760f-2f30-4bd8-b87e-afeb2cd908b9","Type":"ContainerDied","Data":"f6dcc144a57f0363b9c2e85eb031d2ede0f03990713da0d0750cd4fd32b38c08"}
Sep 15 21:30:51.239819 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:51.239424    2564 scope.go:117] "RemoveContainer" containerID="514c068f4887036125ca3d13f50e6392b5bbcf27363e9f5698235db63dc7cf88"
Sep 15 21:30:51.249996 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:51.249975    2564 scope.go:117] "RemoveContainer" containerID="514c068f4887036125ca3d13f50e6392b5bbcf27363e9f5698235db63dc7cf88"
Sep 15 21:30:51.250309 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:30:51.250285    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"514c068f4887036125ca3d13f50e6392b5bbcf27363e9f5698235db63dc7cf88\": container with ID starting with 514c068f4887036125ca3d13f50e6392b5bbcf27363e9f5698235db63dc7cf88 not found: ID does not exist" containerID="514c068f4887036125ca3d13f50e6392b5bbcf27363e9f5698235db63dc7cf88"
Sep 15 21:30:51.250371 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:51.250321    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"514c068f4887036125ca3d13f50e6392b5bbcf27363e9f5698235db63dc7cf88"} err="failed to get container status \"514c068f4887036125ca3d13f50e6392b5bbcf27363e9f5698235db63dc7cf88\": rpc error: code = NotFound desc = could not find container \"514c068f4887036125ca3d13f50e6392b5bbcf27363e9f5698235db63dc7cf88\": container with ID starting with 514c068f4887036125ca3d13f50e6392b5bbcf27363e9f5698235db63dc7cf88 not found: ID does not exist"
Sep 15 21:30:51.260801 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:51.260769    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b6486456d-h4hnf"]
Sep 15 21:30:51.266557 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:51.266526    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7b6486456d-h4hnf"]
Sep 15 21:30:51.972787 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:30:51.972743    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8b7760f-2f30-4bd8-b87e-afeb2cd908b9" path="/var/lib/kubelet/pods/a8b7760f-2f30-4bd8-b87e-afeb2cd908b9/volumes"
Sep 15 21:31:14.342942 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:14.342913    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:31:14.403595 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:14.403549    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:31:50.357827 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:50.357785    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:31:52.046886 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.046850    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7b6486456d-rthtt"]
Sep 15 21:31:52.047283 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.047207    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2beb8a01-4a23-4adf-9ddd-aa76d218527b" containerName="cleanup"
Sep 15 21:31:52.047283 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.047219    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2beb8a01-4a23-4adf-9ddd-aa76d218527b" containerName="cleanup"
Sep 15 21:31:52.047283 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.047232    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8b7760f-2f30-4bd8-b87e-afeb2cd908b9" containerName="manager"
Sep 15 21:31:52.047283 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.047238    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8b7760f-2f30-4bd8-b87e-afeb2cd908b9" containerName="manager"
Sep 15 21:31:52.047283 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.047254    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20a0b584-3f13-44d5-bce5-87e6f46927db" containerName="payload-processing"
Sep 15 21:31:52.047283 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.047259    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="20a0b584-3f13-44d5-bce5-87e6f46927db" containerName="payload-processing"
Sep 15 21:31:52.047283 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.047269    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="deb8a238-e92b-435e-964b-fee577c19776" containerName="payload-processing"
Sep 15 21:31:52.047283 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.047275    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="deb8a238-e92b-435e-964b-fee577c19776" containerName="payload-processing"
Sep 15 21:31:52.047617 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.047329    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="20a0b584-3f13-44d5-bce5-87e6f46927db" containerName="payload-processing"
Sep 15 21:31:52.047617 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.047336    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8b7760f-2f30-4bd8-b87e-afeb2cd908b9" containerName="manager"
Sep 15 21:31:52.047617 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.047344    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2beb8a01-4a23-4adf-9ddd-aa76d218527b" containerName="cleanup"
Sep 15 21:31:52.047617 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.047349    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="deb8a238-e92b-435e-964b-fee577c19776" containerName="payload-processing"
Sep 15 21:31:52.050404 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.050371    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b6486456d-rthtt"
Sep 15 21:31:52.054178 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.054150    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 15 21:31:52.054178 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.054170    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 15 21:31:52.054358 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.054196    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 15 21:31:52.054358 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.054296    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lsxq7\""
Sep 15 21:31:52.061298 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.061272    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b6486456d-rthtt"]
Sep 15 21:31:52.168116 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.168081    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30fcfb68-b033-464b-b73c-8e8d28f8a3ce-webhook-cert\") pod \"maas-controller-7b6486456d-rthtt\" (UID: \"30fcfb68-b033-464b-b73c-8e8d28f8a3ce\") " pod="opendatahub/maas-controller-7b6486456d-rthtt"
Sep 15 21:31:52.168290 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.168138    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30fcfb68-b033-464b-b73c-8e8d28f8a3ce-metrics-certs\") pod \"maas-controller-7b6486456d-rthtt\" (UID: \"30fcfb68-b033-464b-b73c-8e8d28f8a3ce\") " pod="opendatahub/maas-controller-7b6486456d-rthtt"
Sep 15 21:31:52.168290 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.168158    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4hsd\" (UniqueName: \"kubernetes.io/projected/30fcfb68-b033-464b-b73c-8e8d28f8a3ce-kube-api-access-q4hsd\") pod \"maas-controller-7b6486456d-rthtt\" (UID: \"30fcfb68-b033-464b-b73c-8e8d28f8a3ce\") " pod="opendatahub/maas-controller-7b6486456d-rthtt"
Sep 15 21:31:52.269418 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.269372    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30fcfb68-b033-464b-b73c-8e8d28f8a3ce-webhook-cert\") pod \"maas-controller-7b6486456d-rthtt\" (UID: \"30fcfb68-b033-464b-b73c-8e8d28f8a3ce\") " pod="opendatahub/maas-controller-7b6486456d-rthtt"
Sep 15 21:31:52.269636 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.269442    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30fcfb68-b033-464b-b73c-8e8d28f8a3ce-metrics-certs\") pod \"maas-controller-7b6486456d-rthtt\" (UID: \"30fcfb68-b033-464b-b73c-8e8d28f8a3ce\") " pod="opendatahub/maas-controller-7b6486456d-rthtt"
Sep 15 21:31:52.269636 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.269477    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4hsd\" (UniqueName: \"kubernetes.io/projected/30fcfb68-b033-464b-b73c-8e8d28f8a3ce-kube-api-access-q4hsd\") pod \"maas-controller-7b6486456d-rthtt\" (UID: \"30fcfb68-b033-464b-b73c-8e8d28f8a3ce\") " pod="opendatahub/maas-controller-7b6486456d-rthtt"
Sep 15 21:31:52.272155 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.272107    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30fcfb68-b033-464b-b73c-8e8d28f8a3ce-webhook-cert\") pod \"maas-controller-7b6486456d-rthtt\" (UID: \"30fcfb68-b033-464b-b73c-8e8d28f8a3ce\") " pod="opendatahub/maas-controller-7b6486456d-rthtt"
Sep 15 21:31:52.272283 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.272126    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30fcfb68-b033-464b-b73c-8e8d28f8a3ce-metrics-certs\") pod \"maas-controller-7b6486456d-rthtt\" (UID: \"30fcfb68-b033-464b-b73c-8e8d28f8a3ce\") " pod="opendatahub/maas-controller-7b6486456d-rthtt"
Sep 15 21:31:52.278882 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.278860    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4hsd\" (UniqueName: \"kubernetes.io/projected/30fcfb68-b033-464b-b73c-8e8d28f8a3ce-kube-api-access-q4hsd\") pod \"maas-controller-7b6486456d-rthtt\" (UID: \"30fcfb68-b033-464b-b73c-8e8d28f8a3ce\") " pod="opendatahub/maas-controller-7b6486456d-rthtt"
Sep 15 21:31:52.361184 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.361152    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b6486456d-rthtt"
Sep 15 21:31:52.635932 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.635905    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b6486456d-rthtt"]
Sep 15 21:31:52.638055 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:31:52.638027    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30fcfb68_b033_464b_b73c_8e8d28f8a3ce.slice/crio-13472dc65c387a718620a851ea5518bb758d606b0478b4e438ab8c03ab33bdc6 WatchSource:0}: Error finding container 13472dc65c387a718620a851ea5518bb758d606b0478b4e438ab8c03ab33bdc6: Status 404 returned error can't find the container with id 13472dc65c387a718620a851ea5518bb758d606b0478b4e438ab8c03ab33bdc6
Sep 15 21:31:52.639555 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:52.639534    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 21:31:53.548000 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:53.547914    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b6486456d-rthtt" event={"ID":"30fcfb68-b033-464b-b73c-8e8d28f8a3ce","Type":"ContainerStarted","Data":"90e181725c83cfe42a00fa1d9e0d2b9daf5ec5d6780255c0f88e5976ead73646"}
Sep 15 21:31:53.548000 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:53.547972    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b6486456d-rthtt" event={"ID":"30fcfb68-b033-464b-b73c-8e8d28f8a3ce","Type":"ContainerStarted","Data":"13472dc65c387a718620a851ea5518bb758d606b0478b4e438ab8c03ab33bdc6"}
Sep 15 21:31:53.548392 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:53.548014    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7b6486456d-rthtt"
Sep 15 21:31:53.570829 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:31:53.570777    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7b6486456d-rthtt" podStartSLOduration=0.955554445 podStartE2EDuration="1.57076031s" totalImagesPullingTime="615.205865ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:31:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:31:53.569807266 +0000 UTC m=+2050.111969901" watchObservedRunningTime="2026-09-15 21:31:53.57076031 +0000 UTC m=+2050.112922944"
Sep 15 21:32:04.554680 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:04.554646    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7b6486456d-rthtt"
Sep 15 21:32:15.884877 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:15.884832    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:32:15.973052 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:15.973020    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b6486456d-rthtt"]
Sep 15 21:32:15.973448 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:15.973395    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7b6486456d-rthtt" podUID="30fcfb68-b033-464b-b73c-8e8d28f8a3ce" containerName="manager" containerID="cri-o://90e181725c83cfe42a00fa1d9e0d2b9daf5ec5d6780255c0f88e5976ead73646" gracePeriod=30
Sep 15 21:32:16.210017 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:16.209986    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b6486456d-rthtt"
Sep 15 21:32:16.287528 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:16.287478    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4hsd\" (UniqueName: \"kubernetes.io/projected/30fcfb68-b033-464b-b73c-8e8d28f8a3ce-kube-api-access-q4hsd\") pod \"30fcfb68-b033-464b-b73c-8e8d28f8a3ce\" (UID: \"30fcfb68-b033-464b-b73c-8e8d28f8a3ce\") "
Sep 15 21:32:16.287713 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:16.287549    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30fcfb68-b033-464b-b73c-8e8d28f8a3ce-webhook-cert\") pod \"30fcfb68-b033-464b-b73c-8e8d28f8a3ce\" (UID: \"30fcfb68-b033-464b-b73c-8e8d28f8a3ce\") "
Sep 15 21:32:16.287713 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:16.287576    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30fcfb68-b033-464b-b73c-8e8d28f8a3ce-metrics-certs\") pod \"30fcfb68-b033-464b-b73c-8e8d28f8a3ce\" (UID: \"30fcfb68-b033-464b-b73c-8e8d28f8a3ce\") "
Sep 15 21:32:16.289986 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:16.289943    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30fcfb68-b033-464b-b73c-8e8d28f8a3ce-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "30fcfb68-b033-464b-b73c-8e8d28f8a3ce" (UID: "30fcfb68-b033-464b-b73c-8e8d28f8a3ce"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:32:16.289986 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:16.289963    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30fcfb68-b033-464b-b73c-8e8d28f8a3ce-kube-api-access-q4hsd" (OuterVolumeSpecName: "kube-api-access-q4hsd") pod "30fcfb68-b033-464b-b73c-8e8d28f8a3ce" (UID: "30fcfb68-b033-464b-b73c-8e8d28f8a3ce"). InnerVolumeSpecName "kube-api-access-q4hsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:32:16.290194 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:16.289990    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30fcfb68-b033-464b-b73c-8e8d28f8a3ce-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "30fcfb68-b033-464b-b73c-8e8d28f8a3ce" (UID: "30fcfb68-b033-464b-b73c-8e8d28f8a3ce"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:32:16.388738 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:16.388653    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30fcfb68-b033-464b-b73c-8e8d28f8a3ce-webhook-cert\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:32:16.388738 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:16.388684    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30fcfb68-b033-464b-b73c-8e8d28f8a3ce-metrics-certs\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:32:16.388738 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:16.388694    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4hsd\" (UniqueName: \"kubernetes.io/projected/30fcfb68-b033-464b-b73c-8e8d28f8a3ce-kube-api-access-q4hsd\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:32:16.657862 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:16.657772    2564 generic.go:356] "Generic (PLEG): container finished" podID="30fcfb68-b033-464b-b73c-8e8d28f8a3ce" containerID="90e181725c83cfe42a00fa1d9e0d2b9daf5ec5d6780255c0f88e5976ead73646" exitCode=0
Sep 15 21:32:16.657862 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:16.657818    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b6486456d-rthtt"
Sep 15 21:32:16.658050 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:16.657867    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b6486456d-rthtt" event={"ID":"30fcfb68-b033-464b-b73c-8e8d28f8a3ce","Type":"ContainerDied","Data":"90e181725c83cfe42a00fa1d9e0d2b9daf5ec5d6780255c0f88e5976ead73646"}
Sep 15 21:32:16.658050 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:16.657914    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b6486456d-rthtt" event={"ID":"30fcfb68-b033-464b-b73c-8e8d28f8a3ce","Type":"ContainerDied","Data":"13472dc65c387a718620a851ea5518bb758d606b0478b4e438ab8c03ab33bdc6"}
Sep 15 21:32:16.658050 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:16.657936    2564 scope.go:117] "RemoveContainer" containerID="90e181725c83cfe42a00fa1d9e0d2b9daf5ec5d6780255c0f88e5976ead73646"
Sep 15 21:32:16.669537 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:16.669512    2564 scope.go:117] "RemoveContainer" containerID="90e181725c83cfe42a00fa1d9e0d2b9daf5ec5d6780255c0f88e5976ead73646"
Sep 15 21:32:16.670046 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:32:16.670022    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90e181725c83cfe42a00fa1d9e0d2b9daf5ec5d6780255c0f88e5976ead73646\": container with ID starting with 90e181725c83cfe42a00fa1d9e0d2b9daf5ec5d6780255c0f88e5976ead73646 not found: ID does not exist" containerID="90e181725c83cfe42a00fa1d9e0d2b9daf5ec5d6780255c0f88e5976ead73646"
Sep 15 21:32:16.670136 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:16.670058    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90e181725c83cfe42a00fa1d9e0d2b9daf5ec5d6780255c0f88e5976ead73646"} err="failed to get container status \"90e181725c83cfe42a00fa1d9e0d2b9daf5ec5d6780255c0f88e5976ead73646\": rpc error: code = NotFound desc = could not find container \"90e181725c83cfe42a00fa1d9e0d2b9daf5ec5d6780255c0f88e5976ead73646\": container with ID starting with 90e181725c83cfe42a00fa1d9e0d2b9daf5ec5d6780255c0f88e5976ead73646 not found: ID does not exist"
Sep 15 21:32:16.680223 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:16.680195    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b6486456d-rthtt"]
Sep 15 21:32:16.683386 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:16.683365    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7b6486456d-rthtt"]
Sep 15 21:32:17.339653 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:17.339615    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:32:17.435157 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:17.435125    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7b6486456d-xc5gl"]
Sep 15 21:32:17.435525 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:17.435512    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30fcfb68-b033-464b-b73c-8e8d28f8a3ce" containerName="manager"
Sep 15 21:32:17.435587 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:17.435527    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="30fcfb68-b033-464b-b73c-8e8d28f8a3ce" containerName="manager"
Sep 15 21:32:17.435624 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:17.435588    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="30fcfb68-b033-464b-b73c-8e8d28f8a3ce" containerName="manager"
Sep 15 21:32:17.439800 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:17.439779    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b6486456d-xc5gl"
Sep 15 21:32:17.446898 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:17.444351    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lsxq7\""
Sep 15 21:32:17.446898 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:17.444381    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 15 21:32:17.446898 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:17.444579    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 15 21:32:17.446898 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:17.445373    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 15 21:32:17.455237 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:17.455205    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b6486456d-xc5gl"]
Sep 15 21:32:17.497284 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:17.497246    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/972ffd3e-5c74-49dd-abee-3862dbe36fb0-metrics-certs\") pod \"maas-controller-7b6486456d-xc5gl\" (UID: \"972ffd3e-5c74-49dd-abee-3862dbe36fb0\") " pod="opendatahub/maas-controller-7b6486456d-xc5gl"
Sep 15 21:32:17.497284 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:17.497289    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/972ffd3e-5c74-49dd-abee-3862dbe36fb0-webhook-cert\") pod \"maas-controller-7b6486456d-xc5gl\" (UID: \"972ffd3e-5c74-49dd-abee-3862dbe36fb0\") " pod="opendatahub/maas-controller-7b6486456d-xc5gl"
Sep 15 21:32:17.497550 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:17.497412    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdxf4\" (UniqueName: \"kubernetes.io/projected/972ffd3e-5c74-49dd-abee-3862dbe36fb0-kube-api-access-pdxf4\") pod \"maas-controller-7b6486456d-xc5gl\" (UID: \"972ffd3e-5c74-49dd-abee-3862dbe36fb0\") " pod="opendatahub/maas-controller-7b6486456d-xc5gl"
Sep 15 21:32:17.598046 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:17.598010    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdxf4\" (UniqueName: \"kubernetes.io/projected/972ffd3e-5c74-49dd-abee-3862dbe36fb0-kube-api-access-pdxf4\") pod \"maas-controller-7b6486456d-xc5gl\" (UID: \"972ffd3e-5c74-49dd-abee-3862dbe36fb0\") " pod="opendatahub/maas-controller-7b6486456d-xc5gl"
Sep 15 21:32:17.598173 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:17.598065    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/972ffd3e-5c74-49dd-abee-3862dbe36fb0-metrics-certs\") pod \"maas-controller-7b6486456d-xc5gl\" (UID: \"972ffd3e-5c74-49dd-abee-3862dbe36fb0\") " pod="opendatahub/maas-controller-7b6486456d-xc5gl"
Sep 15 21:32:17.598173 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:17.598087    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/972ffd3e-5c74-49dd-abee-3862dbe36fb0-webhook-cert\") pod \"maas-controller-7b6486456d-xc5gl\" (UID: \"972ffd3e-5c74-49dd-abee-3862dbe36fb0\") " pod="opendatahub/maas-controller-7b6486456d-xc5gl"
Sep 15 21:32:17.600792 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:17.600762    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/972ffd3e-5c74-49dd-abee-3862dbe36fb0-metrics-certs\") pod \"maas-controller-7b6486456d-xc5gl\" (UID: \"972ffd3e-5c74-49dd-abee-3862dbe36fb0\") " pod="opendatahub/maas-controller-7b6486456d-xc5gl"
Sep 15 21:32:17.600944 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:17.600912    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/972ffd3e-5c74-49dd-abee-3862dbe36fb0-webhook-cert\") pod \"maas-controller-7b6486456d-xc5gl\" (UID: \"972ffd3e-5c74-49dd-abee-3862dbe36fb0\") " pod="opendatahub/maas-controller-7b6486456d-xc5gl"
Sep 15 21:32:17.610106 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:17.610081    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdxf4\" (UniqueName: \"kubernetes.io/projected/972ffd3e-5c74-49dd-abee-3862dbe36fb0-kube-api-access-pdxf4\") pod \"maas-controller-7b6486456d-xc5gl\" (UID: \"972ffd3e-5c74-49dd-abee-3862dbe36fb0\") " pod="opendatahub/maas-controller-7b6486456d-xc5gl"
Sep 15 21:32:17.757616 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:17.757580    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b6486456d-xc5gl"
Sep 15 21:32:17.972908 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:17.972824    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30fcfb68-b033-464b-b73c-8e8d28f8a3ce" path="/var/lib/kubelet/pods/30fcfb68-b033-464b-b73c-8e8d28f8a3ce/volumes"
Sep 15 21:32:18.033620 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:18.033590    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b6486456d-xc5gl"]
Sep 15 21:32:18.035095 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:32:18.035067    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod972ffd3e_5c74_49dd_abee_3862dbe36fb0.slice/crio-cd2354b5eccb9c5929037503e9970b7d3fac962f6bc611721e2cebc65eaac1f8 WatchSource:0}: Error finding container cd2354b5eccb9c5929037503e9970b7d3fac962f6bc611721e2cebc65eaac1f8: Status 404 returned error can't find the container with id cd2354b5eccb9c5929037503e9970b7d3fac962f6bc611721e2cebc65eaac1f8
Sep 15 21:32:18.669347 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:18.669315    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b6486456d-xc5gl" event={"ID":"972ffd3e-5c74-49dd-abee-3862dbe36fb0","Type":"ContainerStarted","Data":"cd2354b5eccb9c5929037503e9970b7d3fac962f6bc611721e2cebc65eaac1f8"}
Sep 15 21:32:20.681167 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:20.681129    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b6486456d-xc5gl" event={"ID":"972ffd3e-5c74-49dd-abee-3862dbe36fb0","Type":"ContainerStarted","Data":"6abb5a10f7df4b5252324ec1f6cdd3a6b56e79db06ac300de0c4fd4a42d01323"}
Sep 15 21:32:20.681611 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:20.681269    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7b6486456d-xc5gl"
Sep 15 21:32:20.702631 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:20.702579    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7b6486456d-xc5gl" podStartSLOduration=1.947794117 podStartE2EDuration="3.7025624s" totalImagesPullingTime="1.754768283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:32:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:32:20.70075367 +0000 UTC m=+2077.242916331" watchObservedRunningTime="2026-09-15 21:32:20.7025624 +0000 UTC m=+2077.244725035"
Sep 15 21:32:31.688264 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:31.688231    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7b6486456d-xc5gl"
Sep 15 21:32:44.082190 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:44.082162    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-7t46m_c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf/console-operator/1.log"
Sep 15 21:32:44.085648 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:44.085624    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4c7k_fc205950-caf0-4c47-84ce-a8fc54a1afb2/ovn-acl-logging/0.log"
Sep 15 21:32:44.092447 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:44.092426    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-7t46m_c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf/console-operator/1.log"
Sep 15 21:32:44.095559 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:32:44.095537    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4c7k_fc205950-caf0-4c47-84ce-a8fc54a1afb2/ovn-acl-logging/0.log"
Sep 15 21:35:16.898982 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:16.898887    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"]
Sep 15 21:35:16.902609 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:16.902586    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:16.905479 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:16.905455    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-5e63750e-kserve-self-signed-certs\""
Sep 15 21:35:16.916667 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:16.916639    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"]
Sep 15 21:35:16.967741 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:16.967710    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:35:16.997173 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:16.997134    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-dshm\") pod \"e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") " pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:16.997343 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:16.997185    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnm6p\" (UniqueName: \"kubernetes.io/projected/e79302d4-92b6-419d-a36f-33b9d94f3b25-kube-api-access-gnm6p\") pod \"e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") " pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:16.997343 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:16.997256    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-tmp-dir\") pod \"e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") " pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:16.997431 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:16.997340    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-home\") pod \"e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") " pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:16.997431 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:16.997378    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e79302d4-92b6-419d-a36f-33b9d94f3b25-tls-certs\") pod \"e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") " pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:16.997431 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:16.997398    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-model-cache\") pod \"e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") " pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:17.098243 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:17.098201    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-home\") pod \"e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") " pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:17.098243 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:17.098247    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e79302d4-92b6-419d-a36f-33b9d94f3b25-tls-certs\") pod \"e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") " pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:17.098518 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:17.098365    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-model-cache\") pod \"e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") " pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:17.098518 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:17.098425    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-dshm\") pod \"e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") " pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:17.098518 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:17.098476    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnm6p\" (UniqueName: \"kubernetes.io/projected/e79302d4-92b6-419d-a36f-33b9d94f3b25-kube-api-access-gnm6p\") pod \"e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") " pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:17.098707 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:17.098561    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-tmp-dir\") pod \"e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") " pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:17.098707 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:17.098663    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-home\") pod \"e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") " pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:17.098803 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:17.098729    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-model-cache\") pod \"e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") " pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:17.098895 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:17.098876    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-tmp-dir\") pod \"e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") " pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:17.100763 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:17.100736    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-dshm\") pod \"e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") " pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:17.101026 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:17.101008    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e79302d4-92b6-419d-a36f-33b9d94f3b25-tls-certs\") pod \"e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") " pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:17.130794 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:17.130756    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnm6p\" (UniqueName: \"kubernetes.io/projected/e79302d4-92b6-419d-a36f-33b9d94f3b25-kube-api-access-gnm6p\") pod \"e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") " pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:17.213144 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:17.213044    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:17.488622 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:17.488387    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"]
Sep 15 21:35:17.491463 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:35:17.491434    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode79302d4_92b6_419d_a36f_33b9d94f3b25.slice/crio-d67b5fd5ac2bb204b5eceee1ded2ad06b6e0812901b3abb6890ab13dfc56ee3f WatchSource:0}: Error finding container d67b5fd5ac2bb204b5eceee1ded2ad06b6e0812901b3abb6890ab13dfc56ee3f: Status 404 returned error can't find the container with id d67b5fd5ac2bb204b5eceee1ded2ad06b6e0812901b3abb6890ab13dfc56ee3f
Sep 15 21:35:17.542667 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:17.542641    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72" event={"ID":"e79302d4-92b6-419d-a36f-33b9d94f3b25","Type":"ContainerStarted","Data":"d67b5fd5ac2bb204b5eceee1ded2ad06b6e0812901b3abb6890ab13dfc56ee3f"}
Sep 15 21:35:18.549424 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:18.549383    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72" event={"ID":"e79302d4-92b6-419d-a36f-33b9d94f3b25","Type":"ContainerStarted","Data":"50cbabfd2a4fb6362e1a65da8544f833c8f899310f4ce87c1fc50c159ce1712d"}
Sep 15 21:35:18.589335 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:18.589278    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72" podStartSLOduration=2.589259579 podStartE2EDuration="2.589259579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:35:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:35:18.588105994 +0000 UTC m=+2255.130268629" watchObservedRunningTime="2026-09-15 21:35:18.589259579 +0000 UTC m=+2255.131422213"
Sep 15 21:35:27.213484 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:27.213442    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:27.214017 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:27.213521    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:27.226661 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:27.226626    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:27.605535 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:27.605507    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:35:28.903897 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:28.903793    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:35:35.747437 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:35.747404    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:35:52.386375 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:52.386337    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"]
Sep 15 21:35:52.409258 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:52.409206    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:35:54.209709 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:35:54.209676    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:36:07.406847 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.406770    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72" podUID="e79302d4-92b6-419d-a36f-33b9d94f3b25" containerName="main" containerID="cri-o://50cbabfd2a4fb6362e1a65da8544f833c8f899310f4ce87c1fc50c159ce1712d" gracePeriod=45
Sep 15 21:36:07.658124 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.658066    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:36:07.774662 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.774618    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnm6p\" (UniqueName: \"kubernetes.io/projected/e79302d4-92b6-419d-a36f-33b9d94f3b25-kube-api-access-gnm6p\") pod \"e79302d4-92b6-419d-a36f-33b9d94f3b25\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") "
Sep 15 21:36:07.774662 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.774666    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-tmp-dir\") pod \"e79302d4-92b6-419d-a36f-33b9d94f3b25\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") "
Sep 15 21:36:07.774933 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.774721    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-model-cache\") pod \"e79302d4-92b6-419d-a36f-33b9d94f3b25\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") "
Sep 15 21:36:07.774933 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.774764    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-dshm\") pod \"e79302d4-92b6-419d-a36f-33b9d94f3b25\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") "
Sep 15 21:36:07.774933 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.774786    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-home\") pod \"e79302d4-92b6-419d-a36f-33b9d94f3b25\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") "
Sep 15 21:36:07.774933 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.774827    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e79302d4-92b6-419d-a36f-33b9d94f3b25-tls-certs\") pod \"e79302d4-92b6-419d-a36f-33b9d94f3b25\" (UID: \"e79302d4-92b6-419d-a36f-33b9d94f3b25\") "
Sep 15 21:36:07.775119 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.774997    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e79302d4-92b6-419d-a36f-33b9d94f3b25" (UID: "e79302d4-92b6-419d-a36f-33b9d94f3b25"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:36:07.775119 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.775040    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-model-cache" (OuterVolumeSpecName: "model-cache") pod "e79302d4-92b6-419d-a36f-33b9d94f3b25" (UID: "e79302d4-92b6-419d-a36f-33b9d94f3b25"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:36:07.775119 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.775056    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-home" (OuterVolumeSpecName: "home") pod "e79302d4-92b6-419d-a36f-33b9d94f3b25" (UID: "e79302d4-92b6-419d-a36f-33b9d94f3b25"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:36:07.775223 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.775137    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-model-cache\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:36:07.775223 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.775155    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-home\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:36:07.775223 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.775168    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-tmp-dir\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:36:07.777215 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.777180    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-dshm" (OuterVolumeSpecName: "dshm") pod "e79302d4-92b6-419d-a36f-33b9d94f3b25" (UID: "e79302d4-92b6-419d-a36f-33b9d94f3b25"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 21:36:07.777341 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.777236    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e79302d4-92b6-419d-a36f-33b9d94f3b25-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e79302d4-92b6-419d-a36f-33b9d94f3b25" (UID: "e79302d4-92b6-419d-a36f-33b9d94f3b25"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:36:07.777341 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.777279    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e79302d4-92b6-419d-a36f-33b9d94f3b25-kube-api-access-gnm6p" (OuterVolumeSpecName: "kube-api-access-gnm6p") pod "e79302d4-92b6-419d-a36f-33b9d94f3b25" (UID: "e79302d4-92b6-419d-a36f-33b9d94f3b25"). InnerVolumeSpecName "kube-api-access-gnm6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:36:07.796253 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.796220    2564 generic.go:356] "Generic (PLEG): container finished" podID="e79302d4-92b6-419d-a36f-33b9d94f3b25" containerID="50cbabfd2a4fb6362e1a65da8544f833c8f899310f4ce87c1fc50c159ce1712d" exitCode=0
Sep 15 21:36:07.796399 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.796266    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"
Sep 15 21:36:07.796399 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.796308    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72" event={"ID":"e79302d4-92b6-419d-a36f-33b9d94f3b25","Type":"ContainerDied","Data":"50cbabfd2a4fb6362e1a65da8544f833c8f899310f4ce87c1fc50c159ce1712d"}
Sep 15 21:36:07.796399 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.796355    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72" event={"ID":"e79302d4-92b6-419d-a36f-33b9d94f3b25","Type":"ContainerDied","Data":"d67b5fd5ac2bb204b5eceee1ded2ad06b6e0812901b3abb6890ab13dfc56ee3f"}
Sep 15 21:36:07.796399 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.796375    2564 scope.go:117] "RemoveContainer" containerID="50cbabfd2a4fb6362e1a65da8544f833c8f899310f4ce87c1fc50c159ce1712d"
Sep 15 21:36:07.805780 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.805765    2564 scope.go:117] "RemoveContainer" containerID="50cbabfd2a4fb6362e1a65da8544f833c8f899310f4ce87c1fc50c159ce1712d"
Sep 15 21:36:07.806073 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:36:07.806049    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50cbabfd2a4fb6362e1a65da8544f833c8f899310f4ce87c1fc50c159ce1712d\": container with ID starting with 50cbabfd2a4fb6362e1a65da8544f833c8f899310f4ce87c1fc50c159ce1712d not found: ID does not exist" containerID="50cbabfd2a4fb6362e1a65da8544f833c8f899310f4ce87c1fc50c159ce1712d"
Sep 15 21:36:07.806162 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.806079    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50cbabfd2a4fb6362e1a65da8544f833c8f899310f4ce87c1fc50c159ce1712d"} err="failed to get container status \"50cbabfd2a4fb6362e1a65da8544f833c8f899310f4ce87c1fc50c159ce1712d\": rpc error: code = NotFound desc = could not find container \"50cbabfd2a4fb6362e1a65da8544f833c8f899310f4ce87c1fc50c159ce1712d\": container with ID starting with 50cbabfd2a4fb6362e1a65da8544f833c8f899310f4ce87c1fc50c159ce1712d not found: ID does not exist"
Sep 15 21:36:07.824224 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.824197    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"]
Sep 15 21:36:07.832728 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.832699    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72"]
Sep 15 21:36:07.876559 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.876528    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e79302d4-92b6-419d-a36f-33b9d94f3b25-dshm\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:36:07.876559 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.876556    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e79302d4-92b6-419d-a36f-33b9d94f3b25-tls-certs\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:36:07.876559 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.876566    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnm6p\" (UniqueName: \"kubernetes.io/projected/e79302d4-92b6-419d-a36f-33b9d94f3b25-kube-api-access-gnm6p\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:36:07.972932 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:07.972854    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e79302d4-92b6-419d-a36f-33b9d94f3b25" path="/var/lib/kubelet/pods/e79302d4-92b6-419d-a36f-33b9d94f3b25/volumes"
Sep 15 21:36:12.595129 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:12.595074    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="llm/e2e-same-id-a-5e63750e-kserve-67bff85cdc-kxg72" podUID="e79302d4-92b6-419d-a36f-33b9d94f3b25" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Sep 15 21:36:20.622573 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:20.622538    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5cf7bfc64b-2jdtc"]
Sep 15 21:36:20.622958 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:20.622945    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e79302d4-92b6-419d-a36f-33b9d94f3b25" containerName="main"
Sep 15 21:36:20.623004 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:20.622961    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e79302d4-92b6-419d-a36f-33b9d94f3b25" containerName="main"
Sep 15 21:36:20.623039 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:20.623021    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e79302d4-92b6-419d-a36f-33b9d94f3b25" containerName="main"
Sep 15 21:36:20.626036 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:20.626019    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5cf7bfc64b-2jdtc"
Sep 15 21:36:20.637203 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:20.637170    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5cf7bfc64b-2jdtc"]
Sep 15 21:36:20.697709 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:20.697672    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d6eced0-ceed-48d0-9b60-b1c123e875f4-maas-api-tls\") pod \"maas-api-5cf7bfc64b-2jdtc\" (UID: \"4d6eced0-ceed-48d0-9b60-b1c123e875f4\") " pod="odh-ai-gateway-infra/maas-api-5cf7bfc64b-2jdtc"
Sep 15 21:36:20.697891 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:20.697722    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdmt6\" (UniqueName: \"kubernetes.io/projected/4d6eced0-ceed-48d0-9b60-b1c123e875f4-kube-api-access-cdmt6\") pod \"maas-api-5cf7bfc64b-2jdtc\" (UID: \"4d6eced0-ceed-48d0-9b60-b1c123e875f4\") " pod="odh-ai-gateway-infra/maas-api-5cf7bfc64b-2jdtc"
Sep 15 21:36:20.697891 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:20.697796    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d6eced0-ceed-48d0-9b60-b1c123e875f4-openshift-service-ca\") pod \"maas-api-5cf7bfc64b-2jdtc\" (UID: \"4d6eced0-ceed-48d0-9b60-b1c123e875f4\") " pod="odh-ai-gateway-infra/maas-api-5cf7bfc64b-2jdtc"
Sep 15 21:36:20.697891 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:20.697831    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d6eced0-ceed-48d0-9b60-b1c123e875f4-metrics-certs\") pod \"maas-api-5cf7bfc64b-2jdtc\" (UID: \"4d6eced0-ceed-48d0-9b60-b1c123e875f4\") " pod="odh-ai-gateway-infra/maas-api-5cf7bfc64b-2jdtc"
Sep 15 21:36:20.798865 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:20.798828    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d6eced0-ceed-48d0-9b60-b1c123e875f4-maas-api-tls\") pod \"maas-api-5cf7bfc64b-2jdtc\" (UID: \"4d6eced0-ceed-48d0-9b60-b1c123e875f4\") " pod="odh-ai-gateway-infra/maas-api-5cf7bfc64b-2jdtc"
Sep 15 21:36:20.798865 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:20.798875    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdmt6\" (UniqueName: \"kubernetes.io/projected/4d6eced0-ceed-48d0-9b60-b1c123e875f4-kube-api-access-cdmt6\") pod \"maas-api-5cf7bfc64b-2jdtc\" (UID: \"4d6eced0-ceed-48d0-9b60-b1c123e875f4\") " pod="odh-ai-gateway-infra/maas-api-5cf7bfc64b-2jdtc"
Sep 15 21:36:20.799209 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:20.798906    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d6eced0-ceed-48d0-9b60-b1c123e875f4-openshift-service-ca\") pod \"maas-api-5cf7bfc64b-2jdtc\" (UID: \"4d6eced0-ceed-48d0-9b60-b1c123e875f4\") " pod="odh-ai-gateway-infra/maas-api-5cf7bfc64b-2jdtc"
Sep 15 21:36:20.799209 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:20.798925    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d6eced0-ceed-48d0-9b60-b1c123e875f4-metrics-certs\") pod \"maas-api-5cf7bfc64b-2jdtc\" (UID: \"4d6eced0-ceed-48d0-9b60-b1c123e875f4\") " pod="odh-ai-gateway-infra/maas-api-5cf7bfc64b-2jdtc"
Sep 15 21:36:20.799661 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:20.799632    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d6eced0-ceed-48d0-9b60-b1c123e875f4-openshift-service-ca\") pod \"maas-api-5cf7bfc64b-2jdtc\" (UID: \"4d6eced0-ceed-48d0-9b60-b1c123e875f4\") " pod="odh-ai-gateway-infra/maas-api-5cf7bfc64b-2jdtc"
Sep 15 21:36:20.801578 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:20.801553    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d6eced0-ceed-48d0-9b60-b1c123e875f4-metrics-certs\") pod \"maas-api-5cf7bfc64b-2jdtc\" (UID: \"4d6eced0-ceed-48d0-9b60-b1c123e875f4\") " pod="odh-ai-gateway-infra/maas-api-5cf7bfc64b-2jdtc"
Sep 15 21:36:20.801673 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:20.801619    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d6eced0-ceed-48d0-9b60-b1c123e875f4-maas-api-tls\") pod \"maas-api-5cf7bfc64b-2jdtc\" (UID: \"4d6eced0-ceed-48d0-9b60-b1c123e875f4\") " pod="odh-ai-gateway-infra/maas-api-5cf7bfc64b-2jdtc"
Sep 15 21:36:20.808316 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:20.808290    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdmt6\" (UniqueName: \"kubernetes.io/projected/4d6eced0-ceed-48d0-9b60-b1c123e875f4-kube-api-access-cdmt6\") pod \"maas-api-5cf7bfc64b-2jdtc\" (UID: \"4d6eced0-ceed-48d0-9b60-b1c123e875f4\") " pod="odh-ai-gateway-infra/maas-api-5cf7bfc64b-2jdtc"
Sep 15 21:36:20.937837 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:20.937730    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5cf7bfc64b-2jdtc"
Sep 15 21:36:21.217441 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:21.217413    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5cf7bfc64b-2jdtc"]
Sep 15 21:36:21.219377 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:36:21.219348    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d6eced0_ceed_48d0_9b60_b1c123e875f4.slice/crio-28360c40e8d0033ad002579957ae2bdc983cc1658339d9c908b6b388b2c6c7d1 WatchSource:0}: Error finding container 28360c40e8d0033ad002579957ae2bdc983cc1658339d9c908b6b388b2c6c7d1: Status 404 returned error can't find the container with id 28360c40e8d0033ad002579957ae2bdc983cc1658339d9c908b6b388b2c6c7d1
Sep 15 21:36:21.865548 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:21.865516    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5cf7bfc64b-2jdtc" event={"ID":"4d6eced0-ceed-48d0-9b60-b1c123e875f4","Type":"ContainerStarted","Data":"28360c40e8d0033ad002579957ae2bdc983cc1658339d9c908b6b388b2c6c7d1"}
Sep 15 21:36:22.100712 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:22.100681    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:36:22.875044 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:22.875004    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5cf7bfc64b-2jdtc" event={"ID":"4d6eced0-ceed-48d0-9b60-b1c123e875f4","Type":"ContainerStarted","Data":"6fd651cc52ccd0eb71f10c63c01a6dcc4281ce6e63366017b98a6c61fac48811"}
Sep 15 21:36:22.875549 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:22.875072    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5cf7bfc64b-2jdtc"
Sep 15 21:36:22.896455 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:22.896399    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5cf7bfc64b-2jdtc" podStartSLOduration=2.302664279 podStartE2EDuration="2.896382836s" totalImagesPullingTime="593.718557ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 21:36:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:36:22.894594581 +0000 UTC m=+2319.436757227" watchObservedRunningTime="2026-09-15 21:36:22.896382836 +0000 UTC m=+2319.438545513"
Sep 15 21:36:28.885922 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:28.885890    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5cf7bfc64b-2jdtc"
Sep 15 21:36:28.961025 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:28.960985    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"]
Sep 15 21:36:28.961273 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:28.961249    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt" podUID="fcddd559-66df-4dc4-9025-7de984f39336" containerName="maas-api" containerID="cri-o://5c13f8ab877b598dc4ce92a1fc404e1d694bcf1bff4103c86d6517d6f6b1ccd3" gracePeriod=30
Sep 15 21:36:30.225232 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:30.225204    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"
Sep 15 21:36:30.392581 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:30.392538    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcddd559-66df-4dc4-9025-7de984f39336-maas-api-tls\") pod \"fcddd559-66df-4dc4-9025-7de984f39336\" (UID: \"fcddd559-66df-4dc4-9025-7de984f39336\") "
Sep 15 21:36:30.392581 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:30.392587    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84t87\" (UniqueName: \"kubernetes.io/projected/fcddd559-66df-4dc4-9025-7de984f39336-kube-api-access-84t87\") pod \"fcddd559-66df-4dc4-9025-7de984f39336\" (UID: \"fcddd559-66df-4dc4-9025-7de984f39336\") "
Sep 15 21:36:30.392804 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:30.392643    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcddd559-66df-4dc4-9025-7de984f39336-metrics-certs\") pod \"fcddd559-66df-4dc4-9025-7de984f39336\" (UID: \"fcddd559-66df-4dc4-9025-7de984f39336\") "
Sep 15 21:36:30.392804 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:30.392703    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcddd559-66df-4dc4-9025-7de984f39336-openshift-service-ca\") pod \"fcddd559-66df-4dc4-9025-7de984f39336\" (UID: \"fcddd559-66df-4dc4-9025-7de984f39336\") "
Sep 15 21:36:30.393302 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:30.393194    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcddd559-66df-4dc4-9025-7de984f39336-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fcddd559-66df-4dc4-9025-7de984f39336" (UID: "fcddd559-66df-4dc4-9025-7de984f39336"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 21:36:30.394940 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:30.394910    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcddd559-66df-4dc4-9025-7de984f39336-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fcddd559-66df-4dc4-9025-7de984f39336" (UID: "fcddd559-66df-4dc4-9025-7de984f39336"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:36:30.395067 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:30.395004    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcddd559-66df-4dc4-9025-7de984f39336-kube-api-access-84t87" (OuterVolumeSpecName: "kube-api-access-84t87") pod "fcddd559-66df-4dc4-9025-7de984f39336" (UID: "fcddd559-66df-4dc4-9025-7de984f39336"). InnerVolumeSpecName "kube-api-access-84t87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 21:36:30.395144 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:30.395118    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcddd559-66df-4dc4-9025-7de984f39336-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fcddd559-66df-4dc4-9025-7de984f39336" (UID: "fcddd559-66df-4dc4-9025-7de984f39336"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 21:36:30.493461 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:30.493418    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcddd559-66df-4dc4-9025-7de984f39336-maas-api-tls\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:36:30.493671 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:30.493481    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84t87\" (UniqueName: \"kubernetes.io/projected/fcddd559-66df-4dc4-9025-7de984f39336-kube-api-access-84t87\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:36:30.493671 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:30.493514    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcddd559-66df-4dc4-9025-7de984f39336-metrics-certs\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:36:30.493671 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:30.493527    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcddd559-66df-4dc4-9025-7de984f39336-openshift-service-ca\") on node \"ip-10-0-138-221.ec2.internal\" DevicePath \"\""
Sep 15 21:36:30.916834 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:30.916797    2564 generic.go:356] "Generic (PLEG): container finished" podID="fcddd559-66df-4dc4-9025-7de984f39336" containerID="5c13f8ab877b598dc4ce92a1fc404e1d694bcf1bff4103c86d6517d6f6b1ccd3" exitCode=0
Sep 15 21:36:30.917031 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:30.916857    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt" event={"ID":"fcddd559-66df-4dc4-9025-7de984f39336","Type":"ContainerDied","Data":"5c13f8ab877b598dc4ce92a1fc404e1d694bcf1bff4103c86d6517d6f6b1ccd3"}
Sep 15 21:36:30.917031 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:30.916872    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"
Sep 15 21:36:30.917031 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:30.916934    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt" event={"ID":"fcddd559-66df-4dc4-9025-7de984f39336","Type":"ContainerDied","Data":"462a6c45563271aef561edc4a2f270f97d231de660bd51f882468be7638f6dc9"}
Sep 15 21:36:30.917031 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:30.916956    2564 scope.go:117] "RemoveContainer" containerID="5c13f8ab877b598dc4ce92a1fc404e1d694bcf1bff4103c86d6517d6f6b1ccd3"
Sep 15 21:36:30.926765 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:30.926748    2564 scope.go:117] "RemoveContainer" containerID="5c13f8ab877b598dc4ce92a1fc404e1d694bcf1bff4103c86d6517d6f6b1ccd3"
Sep 15 21:36:30.927067 ip-10-0-138-221 kubenswrapper[2564]: E0915 21:36:30.927048    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c13f8ab877b598dc4ce92a1fc404e1d694bcf1bff4103c86d6517d6f6b1ccd3\": container with ID starting with 5c13f8ab877b598dc4ce92a1fc404e1d694bcf1bff4103c86d6517d6f6b1ccd3 not found: ID does not exist" containerID="5c13f8ab877b598dc4ce92a1fc404e1d694bcf1bff4103c86d6517d6f6b1ccd3"
Sep 15 21:36:30.927116 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:30.927076    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c13f8ab877b598dc4ce92a1fc404e1d694bcf1bff4103c86d6517d6f6b1ccd3"} err="failed to get container status \"5c13f8ab877b598dc4ce92a1fc404e1d694bcf1bff4103c86d6517d6f6b1ccd3\": rpc error: code = NotFound desc = could not find container \"5c13f8ab877b598dc4ce92a1fc404e1d694bcf1bff4103c86d6517d6f6b1ccd3\": container with ID starting with 5c13f8ab877b598dc4ce92a1fc404e1d694bcf1bff4103c86d6517d6f6b1ccd3 not found: ID does not exist"
Sep 15 21:36:30.939665 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:30.939637    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"]
Sep 15 21:36:30.943478 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:30.943454    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-646ff496d6-b6hvt"]
Sep 15 21:36:31.978114 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:31.978074    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcddd559-66df-4dc4-9025-7de984f39336" path="/var/lib/kubelet/pods/fcddd559-66df-4dc4-9025-7de984f39336/volumes"
Sep 15 21:36:44.289436 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:44.289360    2564 scope.go:117] "RemoveContainer" containerID="38a9b241a03649cf41543e9930f3a7a8a39112245e1a670bb7dc22e66bc4293f"
Sep 15 21:36:48.424354 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:48.424319    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7b6486456d-xc5gl_972ffd3e-5c74-49dd-abee-3862dbe36fb0/manager/0.log"
Sep 15 21:36:48.888007 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:48.887982    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f85789df9-7zsl6_f105a3b5-d8f2-42cf-ad9e-b82170d48d88/copy-manifests/0.log"
Sep 15 21:36:48.894989 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:48.894959    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f85789df9-7zsl6_f105a3b5-d8f2-42cf-ad9e-b82170d48d88/manager/0.log"
Sep 15 21:36:49.351004 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:49.350971    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5cf7bfc64b-2jdtc_4d6eced0-ceed-48d0-9b60-b1c123e875f4/maas-api/0.log"
Sep 15 21:36:49.825010 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:49.824929    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-wkqcs_d6bfdde1-a751-4785-aa00-00348f766cc9/postgres/0.log"
Sep 15 21:36:50.681666 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:50.681632    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-5n8hg_681809e4-0ba9-46be-adbd-2ee0607431d3/manager/0.log"
Sep 15 21:36:52.155677 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:52.155650    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-cwq7f_bb436781-9ba7-4346-b984-b40c1a82edc6/payload-pre-processing/0.log"
Sep 15 21:36:52.380469 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:52.380425    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-ddcc97c8b-72s7f_34f22718-53a1-41fd-8eda-f8581983bf3d/router/0.log"
Sep 15 21:36:52.709371 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:52.709342    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-wbscl_5791690e-4c5d-4aff-8f8c-e437e809121f/main/0.log"
Sep 15 21:36:52.812287 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:52.812257    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-t44sl_6440c331-fa9c-4264-b61b-c48c30bd2492/main/0.log"
Sep 15 21:36:53.022980 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:53.022901    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-j4s6q_f13a0a9f-8abf-468a-b8b4-77a1057747f1/main/0.log"
Sep 15 21:36:53.238092 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:53.238064    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-gs4d4_4de8b482-cceb-4bec-a1be-b550eba701b0/main/0.log"
Sep 15 21:36:58.741231 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:58.741190    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:36:58.849764 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:36:58.849731    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log" limit=1024
Sep 15 21:37:01.562281 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:01.562247    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pjz7r_88840482-9724-4d2b-8343-3ed2ad33b258/global-pull-secret-syncer/0.log"
Sep 15 21:37:01.648753 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:01.648719    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gwqhk_bc773606-78e0-4ee0-a8ac-29d191e7f936/konnectivity-agent/0.log"
Sep 15 21:37:01.766449 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:01.766414    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-221.ec2.internal_6ea81e5eea0bb90932530ee9266ef4ab/haproxy/0.log"
Sep 15 21:37:06.988416 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:06.988383    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-5n8hg_681809e4-0ba9-46be-adbd-2ee0607431d3/manager/0.log"
Sep 15 21:37:09.034410 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:09.034382    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-qlv8v_d9d3b55f-e9eb-43c9-bc63-0b59e8d52cda/cluster-monitoring-operator/0.log"
Sep 15 21:37:09.224352 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:09.224322    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6pkbm_c59fe925-61ba-436a-b3e0-3de370d5d0e3/node-exporter/0.log"
Sep 15 21:37:09.256086 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:09.256056    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6pkbm_c59fe925-61ba-436a-b3e0-3de370d5d0e3/kube-rbac-proxy/0.log"
Sep 15 21:37:09.277833 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:09.277805    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6pkbm_c59fe925-61ba-436a-b3e0-3de370d5d0e3/init-textfile/0.log"
Sep 15 21:37:11.235412 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.235383    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-2lm2s_ac0aff21-ec17-47fe-b4a2-b90b5d07786b/networking-console-plugin/0.log"
Sep 15 21:37:11.631420 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.631374    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw"]
Sep 15 21:37:11.631881 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.631861    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcddd559-66df-4dc4-9025-7de984f39336" containerName="maas-api"
Sep 15 21:37:11.631978 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.631882    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcddd559-66df-4dc4-9025-7de984f39336" containerName="maas-api"
Sep 15 21:37:11.632032 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.631992    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcddd559-66df-4dc4-9025-7de984f39336" containerName="maas-api"
Sep 15 21:37:11.635422 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.635400    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw"
Sep 15 21:37:11.638286 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.638262    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hwqzn\"/\"kube-root-ca.crt\""
Sep 15 21:37:11.639410 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.639389    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hwqzn\"/\"default-dockercfg-dzqc9\""
Sep 15 21:37:11.639410 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.639406    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hwqzn\"/\"openshift-service-ca.crt\""
Sep 15 21:37:11.644978 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.644956    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw"]
Sep 15 21:37:11.680892 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.680848    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a2d2e20a-6caf-473b-ad48-e9b54ee8bada-podres\") pod \"perf-node-gather-daemonset-grjdw\" (UID: \"a2d2e20a-6caf-473b-ad48-e9b54ee8bada\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw"
Sep 15 21:37:11.681105 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.680942    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-978zt\" (UniqueName: \"kubernetes.io/projected/a2d2e20a-6caf-473b-ad48-e9b54ee8bada-kube-api-access-978zt\") pod \"perf-node-gather-daemonset-grjdw\" (UID: \"a2d2e20a-6caf-473b-ad48-e9b54ee8bada\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw"
Sep 15 21:37:11.681105 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.680983    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a2d2e20a-6caf-473b-ad48-e9b54ee8bada-proc\") pod \"perf-node-gather-daemonset-grjdw\" (UID: \"a2d2e20a-6caf-473b-ad48-e9b54ee8bada\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw"
Sep 15 21:37:11.681105 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.681052    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2d2e20a-6caf-473b-ad48-e9b54ee8bada-sys\") pod \"perf-node-gather-daemonset-grjdw\" (UID: \"a2d2e20a-6caf-473b-ad48-e9b54ee8bada\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw"
Sep 15 21:37:11.681246 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.681110    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2d2e20a-6caf-473b-ad48-e9b54ee8bada-lib-modules\") pod \"perf-node-gather-daemonset-grjdw\" (UID: \"a2d2e20a-6caf-473b-ad48-e9b54ee8bada\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw"
Sep 15 21:37:11.782294 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.782246    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2d2e20a-6caf-473b-ad48-e9b54ee8bada-lib-modules\") pod \"perf-node-gather-daemonset-grjdw\" (UID: \"a2d2e20a-6caf-473b-ad48-e9b54ee8bada\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw"
Sep 15 21:37:11.782294 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.782307    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a2d2e20a-6caf-473b-ad48-e9b54ee8bada-podres\") pod \"perf-node-gather-daemonset-grjdw\" (UID: \"a2d2e20a-6caf-473b-ad48-e9b54ee8bada\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw"
Sep 15 21:37:11.782620 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.782340    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-978zt\" (UniqueName: \"kubernetes.io/projected/a2d2e20a-6caf-473b-ad48-e9b54ee8bada-kube-api-access-978zt\") pod \"perf-node-gather-daemonset-grjdw\" (UID: \"a2d2e20a-6caf-473b-ad48-e9b54ee8bada\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw"
Sep 15 21:37:11.782620 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.782362    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a2d2e20a-6caf-473b-ad48-e9b54ee8bada-proc\") pod \"perf-node-gather-daemonset-grjdw\" (UID: \"a2d2e20a-6caf-473b-ad48-e9b54ee8bada\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw"
Sep 15 21:37:11.782620 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.782396    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2d2e20a-6caf-473b-ad48-e9b54ee8bada-sys\") pod \"perf-node-gather-daemonset-grjdw\" (UID: \"a2d2e20a-6caf-473b-ad48-e9b54ee8bada\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw"
Sep 15 21:37:11.782620 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.782439    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2d2e20a-6caf-473b-ad48-e9b54ee8bada-lib-modules\") pod \"perf-node-gather-daemonset-grjdw\" (UID: \"a2d2e20a-6caf-473b-ad48-e9b54ee8bada\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw"
Sep 15 21:37:11.782620 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.782465    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2d2e20a-6caf-473b-ad48-e9b54ee8bada-sys\") pod \"perf-node-gather-daemonset-grjdw\" (UID: \"a2d2e20a-6caf-473b-ad48-e9b54ee8bada\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw"
Sep 15 21:37:11.782620 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.782481    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a2d2e20a-6caf-473b-ad48-e9b54ee8bada-proc\") pod \"perf-node-gather-daemonset-grjdw\" (UID: \"a2d2e20a-6caf-473b-ad48-e9b54ee8bada\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw"
Sep 15 21:37:11.782620 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.782526    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a2d2e20a-6caf-473b-ad48-e9b54ee8bada-podres\") pod \"perf-node-gather-daemonset-grjdw\" (UID: \"a2d2e20a-6caf-473b-ad48-e9b54ee8bada\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw"
Sep 15 21:37:11.791479 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.791450    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-978zt\" (UniqueName: \"kubernetes.io/projected/a2d2e20a-6caf-473b-ad48-e9b54ee8bada-kube-api-access-978zt\") pod \"perf-node-gather-daemonset-grjdw\" (UID: \"a2d2e20a-6caf-473b-ad48-e9b54ee8bada\") " pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw"
Sep 15 21:37:11.852408 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.852375    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-7t46m_c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf/console-operator/1.log"
Sep 15 21:37:11.856852 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.856828    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-7t46m_c41a4ff2-5904-45fc-89fd-7f8cbd0a9dcf/console-operator/2.log"
Sep 15 21:37:11.946191 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:11.946104    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw"
Sep 15 21:37:12.217556 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:12.217526    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw"]
Sep 15 21:37:12.220103 ip-10-0-138-221 kubenswrapper[2564]: W0915 21:37:12.220075    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda2d2e20a_6caf_473b_ad48_e9b54ee8bada.slice/crio-d95e7bb1a059cb003d18c17bdbfaec44c84c87844e4bdf9cf6cc521139faad34 WatchSource:0}: Error finding container d95e7bb1a059cb003d18c17bdbfaec44c84c87844e4bdf9cf6cc521139faad34: Status 404 returned error can't find the container with id d95e7bb1a059cb003d18c17bdbfaec44c84c87844e4bdf9cf6cc521139faad34
Sep 15 21:37:12.407389 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:12.407351    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-n5nqb_5c896e08-2021-4261-8701-ffb73d0bbed9/download-server/0.log"
Sep 15 21:37:13.057561 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:13.057530    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-dbwkr_26e0bf54-90c8-4daa-9808-16b7501bc0a3/volume-data-source-validator/0.log"
Sep 15 21:37:13.132786 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:13.132740    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw" event={"ID":"a2d2e20a-6caf-473b-ad48-e9b54ee8bada","Type":"ContainerStarted","Data":"5825190a91a5905c30e8f33885edc3a05cb22689b622f92df1c85bb2684e1685"}
Sep 15 21:37:13.132967 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:13.132794    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw" event={"ID":"a2d2e20a-6caf-473b-ad48-e9b54ee8bada","Type":"ContainerStarted","Data":"d95e7bb1a059cb003d18c17bdbfaec44c84c87844e4bdf9cf6cc521139faad34"}
Sep 15 21:37:13.132967 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:13.132901    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw"
Sep 15 21:37:13.149196 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:13.149117    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw" podStartSLOduration=2.149098627 podStartE2EDuration="2.149098627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 21:37:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 21:37:13.148990036 +0000 UTC m=+2369.691152697" watchObservedRunningTime="2026-09-15 21:37:13.149098627 +0000 UTC m=+2369.691261263"
Sep 15 21:37:14.214352 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:14.214327    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rqmrf_04865999-e56e-4a65-9c4a-b52647e7b1db/dns/0.log"
Sep 15 21:37:14.239052 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:14.239024    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rqmrf_04865999-e56e-4a65-9c4a-b52647e7b1db/kube-rbac-proxy/0.log"
Sep 15 21:37:14.261076 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:14.261050    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mt7b6_7c4d34d1-b3a5-4974-9db9-0ccf8786db47/dns-node-resolver/0.log"
Sep 15 21:37:14.938045 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:14.938017    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cdrtr_afadf0fd-9154-4a32-8abe-d8b29dfcaa90/node-ca/0.log"
Sep 15 21:37:16.353461 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:16.353431    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-cwq7f_bb436781-9ba7-4346-b984-b40c1a82edc6/payload-pre-processing/0.log"
Sep 15 21:37:16.403564 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:16.403536    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-ddcc97c8b-72s7f_34f22718-53a1-41fd-8eda-f8581983bf3d/router/0.log"
Sep 15 21:37:17.299008 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:17.298979    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5mvsw_2aae8637-fdbb-4c3c-a878-ec6ea9a25c7b/serve-healthcheck-canary/0.log"
Sep 15 21:37:17.819695 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:17.819655    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-rlf4s_4ffff173-a298-4e73-98e3-d77f1c0a11e8/insights-operator/0.log"
Sep 15 21:37:17.946534 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:17.946507    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xc4sd_8cba866d-6b74-4efa-802d-3d4248a87202/kube-rbac-proxy/0.log"
Sep 15 21:37:17.971975 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:17.971950    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xc4sd_8cba866d-6b74-4efa-802d-3d4248a87202/exporter/0.log"
Sep 15 21:37:18.001474 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:18.001448    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xc4sd_8cba866d-6b74-4efa-802d-3d4248a87202/extractor/0.log"
Sep 15 21:37:19.148677 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:19.148647    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hwqzn/perf-node-gather-daemonset-grjdw"
Sep 15 21:37:20.790003 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:20.789902    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7b6486456d-xc5gl_972ffd3e-5c74-49dd-abee-3862dbe36fb0/manager/0.log"
Sep 15 21:37:20.981958 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:20.981915    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f85789df9-7zsl6_f105a3b5-d8f2-42cf-ad9e-b82170d48d88/manager/0.log"
Sep 15 21:37:21.002102 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:21.002075    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f85789df9-7zsl6_f105a3b5-d8f2-42cf-ad9e-b82170d48d88/copy-manifests/0.log"
Sep 15 21:37:22.374616 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:22.374580    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5665dbb77c-zzs2w_8bc5b527-c655-4c33-a3b0-cf77e0660ac3/manager/0.log"
Sep 15 21:37:22.654899 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:22.654818    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-cwq7f_bb436781-9ba7-4346-b984-b40c1a82edc6/payload-pre-processing/0.log"
Sep 15 21:37:22.769862 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:22.769833    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-ddcc97c8b-72s7f_34f22718-53a1-41fd-8eda-f8581983bf3d/router/0.log"
Sep 15 21:37:24.712099 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:24.712055    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-cwq7f_bb436781-9ba7-4346-b984-b40c1a82edc6/payload-pre-processing/0.log"
Sep 15 21:37:24.756799 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:24.756766    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-ddcc97c8b-72s7f_34f22718-53a1-41fd-8eda-f8581983bf3d/router/0.log"
Sep 15 21:37:27.719515 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:27.719407    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-snq76_d6e9962c-32fc-4fd1-8baa-aed7a396f606/migrator/0.log"
Sep 15 21:37:27.740281 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:27.740246    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-snq76_d6e9962c-32fc-4fd1-8baa-aed7a396f606/graceful-termination/0.log"
Sep 15 21:37:28.077965 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:28.077875    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-ppbrv_f3b99f56-873f-40d8-abf3-b0c02523b2f7/kube-storage-version-migrator-operator/1.log"
Sep 15 21:37:28.078900 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:28.078880    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-ppbrv_f3b99f56-873f-40d8-abf3-b0c02523b2f7/kube-storage-version-migrator-operator/0.log"
Sep 15 21:37:29.094083 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:29.094030    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b8vrp_b24f6335-59f4-4b7a-8548-56465ed60966/kube-multus-additional-cni-plugins/0.log"
Sep 15 21:37:29.113971 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:29.113946    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b8vrp_b24f6335-59f4-4b7a-8548-56465ed60966/egress-router-binary-copy/0.log"
Sep 15 21:37:29.134111 ip-10-0-138-221 kubenswrapper[2564]: I0915 21:37:29.134084    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b8vrp_b24f6335-59f4-4b7a-8548-56465ed60966/cni-plugins/0.log"