Sep 07 03:46:30.156445 ip-10-0-132-175 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 07 03:46:30.156461 ip-10-0-132-175 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 07 03:46:30.156471 ip-10-0-132-175 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 07 03:46:30.156799 ip-10-0-132-175 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 07 03:46:40.405325 ip-10-0-132-175 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 07 03:46:40.405346 ip-10-0-132-175 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 8ef172da971e45f6a940ff6b23364344 --
Sep 07 03:48:32.959578 ip-10-0-132-175 systemd[1]: Starting Kubernetes Kubelet...
Sep 07 03:48:33.322372 ip-10-0-132-175 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 03:48:33.322372 ip-10-0-132-175 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 07 03:48:33.322372 ip-10-0-132-175 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 03:48:33.322372 ip-10-0-132-175 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 07 03:48:33.322372 ip-10-0-132-175 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 03:48:33.324897 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.324810    2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 07 03:48:33.328474 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328458    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 03:48:33.328474 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328475    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 03:48:33.328540 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328479    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 03:48:33.328540 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328483    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 03:48:33.328540 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328486    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 03:48:33.328540 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328489    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 03:48:33.328540 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328492    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 03:48:33.328540 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328495    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 03:48:33.328540 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328498    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 03:48:33.328540 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328500    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 03:48:33.328540 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328503    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 03:48:33.328540 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328506    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 03:48:33.328540 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328509    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 03:48:33.328540 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328511    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 03:48:33.328540 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328514    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 03:48:33.328540 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328516    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 03:48:33.328540 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328519    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 03:48:33.328540 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328524    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 03:48:33.328540 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328527    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 03:48:33.328540 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328530    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 03:48:33.328540 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328532    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 03:48:33.328540 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328535    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 03:48:33.329022 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328538    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 03:48:33.329022 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328540    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 03:48:33.329022 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328543    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 03:48:33.329022 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328546    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 03:48:33.329022 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328549    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 03:48:33.329022 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328552    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 03:48:33.329022 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328555    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 03:48:33.329022 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328558    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 03:48:33.329022 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328561    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 03:48:33.329022 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328564    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 03:48:33.329022 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328567    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 03:48:33.329022 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328569    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 03:48:33.329022 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328572    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 03:48:33.329022 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328575    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 03:48:33.329022 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328577    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 03:48:33.329022 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328580    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 03:48:33.329022 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328582    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 03:48:33.329022 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328585    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 03:48:33.329022 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328587    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 03:48:33.329022 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328589    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 03:48:33.329022 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328592    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 03:48:33.329545 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328594    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 03:48:33.329545 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328597    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 03:48:33.329545 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328599    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 03:48:33.329545 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328602    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 03:48:33.329545 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328604    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 03:48:33.329545 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328607    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 03:48:33.329545 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328609    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 03:48:33.329545 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328612    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 03:48:33.329545 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328614    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 03:48:33.329545 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328617    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 03:48:33.329545 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328619    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 03:48:33.329545 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328622    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 03:48:33.329545 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328625    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 03:48:33.329545 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328628    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 03:48:33.329545 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328631    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 03:48:33.329545 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328633    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 03:48:33.329545 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328635    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 03:48:33.329545 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328638    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 03:48:33.329545 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328640    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 03:48:33.329545 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328643    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 03:48:33.330071 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328646    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 03:48:33.330071 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328648    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 03:48:33.330071 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328651    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 03:48:33.330071 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328656    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 03:48:33.330071 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328659    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 03:48:33.330071 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328662    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 03:48:33.330071 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328664    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 03:48:33.330071 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328667    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 03:48:33.330071 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328670    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 03:48:33.330071 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328673    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 03:48:33.330071 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328675    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 03:48:33.330071 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328678    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 03:48:33.330071 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328680    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 03:48:33.330071 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328683    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 03:48:33.330071 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328685    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 03:48:33.330071 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328688    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 03:48:33.330071 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328691    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 03:48:33.330071 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328693    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 03:48:33.330071 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328707    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 03:48:33.330544 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328710    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 03:48:33.330544 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328717    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 03:48:33.330544 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328720    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 03:48:33.330544 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328722    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 03:48:33.330544 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328725    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 03:48:33.330544 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328728    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 03:48:33.330544 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328731    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 03:48:33.330544 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328734    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 03:48:33.330544 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328736    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 03:48:33.330544 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328739    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 03:48:33.330544 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328741    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 03:48:33.330544 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328744    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 03:48:33.330544 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328748    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 03:48:33.330544 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328751    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 03:48:33.330544 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328754    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 03:48:33.330544 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328757    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 03:48:33.330544 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.328760    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 03:48:33.330544 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329459    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 03:48:33.330544 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329467    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 03:48:33.331005 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329472    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 03:48:33.331005 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329475    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 03:48:33.331005 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329478    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 03:48:33.331005 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329481    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 03:48:33.331005 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329483    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 03:48:33.331005 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329486    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 03:48:33.331005 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329489    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 03:48:33.331005 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329492    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 03:48:33.331005 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329494    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 03:48:33.331005 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329497    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 03:48:33.331005 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329500    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 03:48:33.331005 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329503    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 03:48:33.331005 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329505    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 03:48:33.331005 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329508    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 03:48:33.331005 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329511    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 03:48:33.331005 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329514    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 03:48:33.331005 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329517    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 03:48:33.331005 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329520    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 03:48:33.331005 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329523    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 03:48:33.331511 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329527    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 03:48:33.331511 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329530    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 03:48:33.331511 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329533    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 03:48:33.331511 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329536    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 03:48:33.331511 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329538    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 03:48:33.331511 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329541    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 03:48:33.331511 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329544    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 03:48:33.331511 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329546    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 03:48:33.331511 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329549    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 03:48:33.331511 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329552    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 03:48:33.331511 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329554    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 03:48:33.331511 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329557    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 03:48:33.331511 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329559    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 03:48:33.331511 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329562    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 03:48:33.331511 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329565    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 03:48:33.331511 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329568    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 03:48:33.331511 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329571    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 03:48:33.331511 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329574    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 03:48:33.331511 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329576    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 03:48:33.332073 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329579    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 03:48:33.332073 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329581    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 03:48:33.332073 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329583    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 03:48:33.332073 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329586    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 03:48:33.332073 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329589    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 03:48:33.332073 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329591    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 03:48:33.332073 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329594    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 03:48:33.332073 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329596    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 03:48:33.332073 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329599    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 03:48:33.332073 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329602    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 03:48:33.332073 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329604    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 03:48:33.332073 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329607    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 03:48:33.332073 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329610    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 03:48:33.332073 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329613    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 03:48:33.332073 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329615    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 03:48:33.332073 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329618    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 03:48:33.332073 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329620    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 03:48:33.332073 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329623    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 03:48:33.332073 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329625    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 03:48:33.332073 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329628    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 03:48:33.332590 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329630    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 03:48:33.332590 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329633    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 03:48:33.332590 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329635    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 03:48:33.332590 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329637    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 03:48:33.332590 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329640    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 03:48:33.332590 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329642    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 03:48:33.332590 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329645    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 03:48:33.332590 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329649    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 03:48:33.332590 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329651    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 03:48:33.332590 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329654    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 03:48:33.332590 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329656    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 03:48:33.332590 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329659    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 03:48:33.332590 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329661    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 03:48:33.332590 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329664    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 03:48:33.332590 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329666    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 03:48:33.332590 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329668    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 03:48:33.332590 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329672    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 03:48:33.332590 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329674    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 03:48:33.332590 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329677    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 03:48:33.332590 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329679    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 03:48:33.333092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329682    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 03:48:33.333092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329684    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 03:48:33.333092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329686    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 03:48:33.333092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329690    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 03:48:33.333092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329693    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 03:48:33.333092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329697    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 03:48:33.333092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329699    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 03:48:33.333092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329702    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 03:48:33.333092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329704    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 03:48:33.333092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329707    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 03:48:33.333092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329709    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 03:48:33.333092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329712    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 03:48:33.333092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329714    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 03:48:33.333092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329717    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 03:48:33.333092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329720    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 03:48:33.333092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329722    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 03:48:33.333092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329724    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 03:48:33.333092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329727    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 03:48:33.333092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.329729    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 03:48:33.333092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330159    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 03:48:33.333601 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330165    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 03:48:33.333601 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330168    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 03:48:33.333601 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330171    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 03:48:33.333601 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330174    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 03:48:33.333601 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330178    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 03:48:33.333601 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330181    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 03:48:33.333601 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330183    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 03:48:33.333601 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330186    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 03:48:33.333601 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330189    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 03:48:33.333601 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330192    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 03:48:33.333601 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330195    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 03:48:33.333601 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330198    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 03:48:33.333601 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330201    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 03:48:33.333601 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330203    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 03:48:33.333601 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330206    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 03:48:33.333601 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330226    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 03:48:33.333601 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330231    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 03:48:33.333601 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330235    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 03:48:33.333601 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330238    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 03:48:33.334067 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330241    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 03:48:33.334067 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330243    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 03:48:33.334067 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330246    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 03:48:33.334067 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330249    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 03:48:33.334067 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330251    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 03:48:33.334067 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330254    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 03:48:33.334067 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330256    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 03:48:33.334067 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330259    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 03:48:33.334067 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330261    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 03:48:33.334067 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330264    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 03:48:33.334067 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330266    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 03:48:33.334067 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330269    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 03:48:33.334067 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330271    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 03:48:33.334067 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330274    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 03:48:33.334067 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330277    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 03:48:33.334067 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330279    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 03:48:33.334067 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330281    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 03:48:33.334067 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330285    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 03:48:33.334067 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330288    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 03:48:33.334067 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330291    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 03:48:33.334570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330293    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 03:48:33.334570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330296    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 03:48:33.334570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330298    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 03:48:33.334570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330301    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 03:48:33.334570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330303    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 03:48:33.334570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330306    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 03:48:33.334570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330308    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 03:48:33.334570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330311    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 03:48:33.334570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330315    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 03:48:33.334570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330318    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 03:48:33.334570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330320    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 03:48:33.334570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330323    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 03:48:33.334570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330325    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 03:48:33.334570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330327    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 03:48:33.334570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330330    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 03:48:33.334570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330333    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 03:48:33.334570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330335    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 03:48:33.334570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330338    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 03:48:33.334570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330340    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 03:48:33.334570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330343    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 03:48:33.335124 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330346    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 03:48:33.335124 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330348    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 03:48:33.335124 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330351    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 03:48:33.335124 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330353    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 03:48:33.335124 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330356    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 03:48:33.335124 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330358    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 03:48:33.335124 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330361    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 03:48:33.335124 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330363    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 03:48:33.335124 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330366    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 03:48:33.335124 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330368    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 03:48:33.335124 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330371    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 03:48:33.335124 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330374    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 03:48:33.335124 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330376    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 03:48:33.335124 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330379    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 03:48:33.335124 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330382    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 03:48:33.335124 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330384    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 03:48:33.335124 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330387    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 03:48:33.335124 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330389    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 03:48:33.335124 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330392    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 03:48:33.335124 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330396    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 03:48:33.335635 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330400    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 03:48:33.335635 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330402    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 03:48:33.335635 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330405    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 03:48:33.335635 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330407    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 03:48:33.335635 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330410    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 03:48:33.335635 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330413    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 03:48:33.335635 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330417    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 03:48:33.335635 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330420    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 03:48:33.335635 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330422    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 03:48:33.335635 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330424    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 03:48:33.335635 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330427    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 03:48:33.335635 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330429    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 03:48:33.335635 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330432    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 03:48:33.335635 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330434    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 03:48:33.335635 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330437    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 03:48:33.335635 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330439    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 03:48:33.335635 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330442    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 03:48:33.335635 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330445    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 03:48:33.335635 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.330449    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330521    2583 flags.go:64] FLAG: --address="0.0.0.0"
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330551    2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330557    2583 flags.go:64] FLAG: --anonymous-auth="true"
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330562    2583 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330566    2583 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330570    2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330574    2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330578    2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330581    2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330584    2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330587    2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330590    2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330593    2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330597    2583 flags.go:64] FLAG: --cgroup-root=""
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330600    2583 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330603    2583 flags.go:64] FLAG: --client-ca-file=""
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330606    2583 flags.go:64] FLAG: --cloud-provider="external"
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330609    2583 flags.go:64] FLAG: --cluster-dns="[]"
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330614    2583 flags.go:64] FLAG: --cluster-domain=""
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330617    2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330620    2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330623    2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330626    2583 flags.go:64] FLAG: --container-log-max-files="5"
Sep 07 03:48:33.336081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330636    2583 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330639    2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330642    2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330646    2583 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330649    2583 flags.go:64] FLAG: --contention-profiling="false"
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330652    2583 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330655    2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330659    2583 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330662    2583 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330666    2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330670    2583 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330673    2583 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330676    2583 flags.go:64] FLAG: --enable-load-reader="false"
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330679    2583 flags.go:64] FLAG: --enable-server="true"
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330682    2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330686    2583 flags.go:64] FLAG: --event-burst="100"
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330690    2583 flags.go:64] FLAG: --event-qps="50"
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330692    2583 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330696    2583 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330698    2583 flags.go:64] FLAG: --eviction-hard=""
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330702    2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330705    2583 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330708    2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330711    2583 flags.go:64] FLAG: --eviction-soft=""
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330714    2583 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 07 03:48:33.336658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330717    2583 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330721    2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330724    2583 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330727    2583 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330730    2583 flags.go:64] FLAG: --fail-swap-on="true"
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330732    2583 flags.go:64] FLAG: --feature-gates=""
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330736    2583 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330739    2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330742    2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330745    2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330748    2583 flags.go:64] FLAG: --healthz-port="10248"
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330751    2583 flags.go:64] FLAG: --help="false"
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330754    2583 flags.go:64] FLAG: --hostname-override="ip-10-0-132-175.ec2.internal"
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330756    2583 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330759    2583 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330762    2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330765    2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330770    2583 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330773    2583 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330776    2583 flags.go:64] FLAG: --image-service-endpoint=""
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330778    2583 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330781    2583 flags.go:64] FLAG: --kube-api-burst="100"
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330784    2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330788    2583 flags.go:64] FLAG: --kube-api-qps="50"
Sep 07 03:48:33.337298 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330790    2583 flags.go:64] FLAG: --kube-reserved=""
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330793    2583 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330796    2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330799    2583 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330802    2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330804    2583 flags.go:64] FLAG: --lock-file=""
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330807    2583 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330810    2583 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330813    2583 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330818    2583 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330821    2583 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330824    2583 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330826    2583 flags.go:64] FLAG: --logging-format="text"
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330829    2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330832    2583 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330835    2583 flags.go:64] FLAG: --manifest-url=""
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330838    2583 flags.go:64] FLAG: --manifest-url-header=""
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330842    2583 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330845    2583 flags.go:64] FLAG: --max-open-files="1000000"
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330849    2583 flags.go:64] FLAG: --max-pods="110"
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330852    2583 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330855    2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330858    2583 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330861    2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330863    2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 07 03:48:33.337893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330867    2583 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330870    2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330878    2583 flags.go:64] FLAG: --node-status-max-images="50"
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330881    2583 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330884    2583 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330887    2583 flags.go:64] FLAG: --pod-cidr=""
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330890    2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330895    2583 flags.go:64] FLAG: --pod-manifest-path=""
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330898    2583 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330901    2583 flags.go:64] FLAG: --pods-per-core="0"
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330904    2583 flags.go:64] FLAG: --port="10250"
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330907    2583 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330910    2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ff0bc8721e4a3a2c"
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330913    2583 flags.go:64] FLAG: --qos-reserved=""
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330916    2583 flags.go:64] FLAG: --read-only-port="10255"
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330919    2583 flags.go:64] FLAG: --register-node="true"
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330922    2583 flags.go:64] FLAG: --register-with-taints=""
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330925    2583 flags.go:64] FLAG: --registry-burst="10"
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330929    2583 flags.go:64] FLAG: --registry-qps="5"
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330932    2583 flags.go:64] FLAG: --reserved-cpus=""
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330935    2583 flags.go:64] FLAG: --reserved-memory=""
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330938    2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330941    2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330944    2583 flags.go:64] FLAG: --rotate-certificates="false"
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330947    2583 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 07 03:48:33.338769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330950    2583 flags.go:64] FLAG: --runonce="false"
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330952    2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330955    2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330959    2583 flags.go:64] FLAG: --seccomp-default="false"
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330961    2583 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330964    2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330967    2583 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330970    2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330974    2583 flags.go:64] FLAG: --storage-driver-password="root"
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330977    2583 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330981    2583 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330984    2583 flags.go:64] FLAG: --storage-driver-user="root"
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330986    2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330989    2583 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330992    2583 flags.go:64] FLAG: --system-cgroups=""
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.330995    2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.331000    2583 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.331003    2583 flags.go:64] FLAG: --tls-cert-file=""
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.331006    2583 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.331011    2583 flags.go:64] FLAG: --tls-min-version=""
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.331014    2583 flags.go:64] FLAG: --tls-private-key-file=""
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.331016    2583 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.331019    2583 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.331022    2583 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.331025    2583 flags.go:64] FLAG: --v="2"
Sep 07 03:48:33.339584 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.331029    2583 flags.go:64] FLAG: --version="false"
Sep 07 03:48:33.340206 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.331033    2583 flags.go:64] FLAG: --vmodule=""
Sep 07 03:48:33.340206 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.331037    2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 07 03:48:33.340206 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.331041    2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 07 03:48:33.340206 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331152    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 03:48:33.340206 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331155    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 03:48:33.340206 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331158    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 03:48:33.340206 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331161    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 03:48:33.340206 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331163    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 03:48:33.340206 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331166    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 03:48:33.340206 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331168    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 03:48:33.340206 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331170    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 03:48:33.340206 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331174    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 03:48:33.340206 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331176    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 03:48:33.340206 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331179    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 03:48:33.340206 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331182    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 03:48:33.340206 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331185    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 03:48:33.340206 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331189    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 03:48:33.340206 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331192    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 03:48:33.340206 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331194    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 03:48:33.340206 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331197    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 03:48:33.340206 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331199    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 03:48:33.340752 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331202    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 03:48:33.340752 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331205    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 03:48:33.340752 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331221    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 03:48:33.340752 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331224    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 03:48:33.340752 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331227    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 03:48:33.340752 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331229    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 03:48:33.340752 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331232    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 03:48:33.340752 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331234    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 03:48:33.340752 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331237    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 03:48:33.340752 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331240    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 03:48:33.340752 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331242    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 03:48:33.340752 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331245    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 03:48:33.340752 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331248    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 03:48:33.340752 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331250    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 03:48:33.340752 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331253    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 03:48:33.340752 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331255    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 03:48:33.340752 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331258    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 03:48:33.340752 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331261    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 03:48:33.340752 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331263    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 03:48:33.341203 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331266    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 03:48:33.341203 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331268    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 03:48:33.341203 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331271    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 03:48:33.341203 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331273    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 03:48:33.341203 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331276    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 03:48:33.341203 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331278    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 03:48:33.341203 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331284    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 03:48:33.341203 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331288    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 03:48:33.341203 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331292    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 03:48:33.341203 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331295    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 03:48:33.341203 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331298    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 03:48:33.341203 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331301    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 03:48:33.341203 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331304    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 03:48:33.341203 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331307    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 03:48:33.341203 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331310    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 03:48:33.341203 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331312    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 03:48:33.341203 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331315    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 03:48:33.341203 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331317    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 03:48:33.341203 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331320    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 03:48:33.341203 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331323    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 03:48:33.341712 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331325    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 03:48:33.341712 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331328    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 03:48:33.341712 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331330    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 03:48:33.341712 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331333    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 03:48:33.341712 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331335    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 03:48:33.341712 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331338    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 03:48:33.341712 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331341    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 03:48:33.341712 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331343    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 03:48:33.341712 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331346    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 03:48:33.341712 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331349    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 03:48:33.341712 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331353    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 03:48:33.341712 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331356    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 03:48:33.341712 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331359    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 03:48:33.341712 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331361    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 03:48:33.341712 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331364    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 03:48:33.341712 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331367    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 03:48:33.341712 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331369    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 03:48:33.341712 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331372    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 03:48:33.341712 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331376    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 03:48:33.341712 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331378    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 03:48:33.342227 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331382    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 03:48:33.342227 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331385    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 03:48:33.342227 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331387    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 03:48:33.342227 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331389    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 03:48:33.342227 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331392    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 03:48:33.342227 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331394    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 03:48:33.342227 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331397    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 03:48:33.342227 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331400    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 03:48:33.342227 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331402    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 03:48:33.342227 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331405    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 03:48:33.342227 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331407    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 03:48:33.342227 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331410    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 03:48:33.342227 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331413    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 03:48:33.342227 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331415    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 03:48:33.342227 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331418    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 03:48:33.342227 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331420    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 03:48:33.342227 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331423    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 03:48:33.342227 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331425    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 03:48:33.342227 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331428    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 03:48:33.342227 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331431    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 03:48:33.342748 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331435    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 03:48:33.342748 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.331438    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 03:48:33.342748 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.332039    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 03:48:33.342748 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.340932    2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 07 03:48:33.342748 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.340951    2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 07 03:48:33.342748 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.340971    2583 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 07 03:48:33.342748 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.340977    2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 07 03:48:33.342748 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341045    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 03:48:33.342748 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341051    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 03:48:33.342748 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341054    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 03:48:33.342748 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341058    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 03:48:33.342748 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341061    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 03:48:33.342748 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341064    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 03:48:33.342748 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341067    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 03:48:33.343092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341069    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 03:48:33.343092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341073    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 03:48:33.343092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341075    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 03:48:33.343092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341080    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 03:48:33.343092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341086    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 03:48:33.343092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341090    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 03:48:33.343092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341095    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 03:48:33.343092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341098    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 03:48:33.343092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341102    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 03:48:33.343092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341104    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 03:48:33.343092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341107    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 03:48:33.343092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341110    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 03:48:33.343092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341112    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 03:48:33.343092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341115    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 03:48:33.343092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341117    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 03:48:33.343092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341120    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 03:48:33.343092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341122    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 03:48:33.343092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341125    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 03:48:33.343092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341127    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 03:48:33.343092 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341130    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 03:48:33.343593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341133    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 03:48:33.343593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341136    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 03:48:33.343593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341139    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 03:48:33.343593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341142    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 03:48:33.343593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341145    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 03:48:33.343593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341148    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 03:48:33.343593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341150    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 03:48:33.343593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341153    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 03:48:33.343593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341156    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 03:48:33.343593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341159    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 03:48:33.343593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341163    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 03:48:33.343593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341168    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 03:48:33.343593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341172    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 03:48:33.343593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341176    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 03:48:33.343593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341179    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 03:48:33.343593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341182    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 03:48:33.343593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341185    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 03:48:33.343593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341188    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 03:48:33.343593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341190    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 03:48:33.343593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341193    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 03:48:33.344095 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341196    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 03:48:33.344095 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341198    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 03:48:33.344095 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341202    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 03:48:33.344095 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341207    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 03:48:33.344095 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341224    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 03:48:33.344095 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341227    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 03:48:33.344095 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341230    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 03:48:33.344095 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341233    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 03:48:33.344095 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341235    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 03:48:33.344095 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341239    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 03:48:33.344095 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341243    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 03:48:33.344095 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341248    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 03:48:33.344095 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341252    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 03:48:33.344095 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341257    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 03:48:33.344095 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341260    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 03:48:33.344095 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341264    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 03:48:33.344095 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341266    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 03:48:33.344095 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341269    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 03:48:33.344095 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341272    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 03:48:33.344095 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341274    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 03:48:33.344600 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341277    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 03:48:33.344600 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341280    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 03:48:33.344600 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341283    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 03:48:33.344600 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341285    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 03:48:33.344600 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341288    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 03:48:33.344600 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341290    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 03:48:33.344600 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341293    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 03:48:33.344600 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341295    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 03:48:33.344600 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341298    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 03:48:33.344600 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341300    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 03:48:33.344600 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341303    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 03:48:33.344600 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341305    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 03:48:33.344600 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341309    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 03:48:33.344600 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341311    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 03:48:33.344600 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341316    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 03:48:33.344600 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341321    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 03:48:33.344600 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341326    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 03:48:33.344600 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341332    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 03:48:33.344600 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341336    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 03:48:33.344600 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341339    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 03:48:33.345091 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341342    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 03:48:33.345091 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341345    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 03:48:33.345091 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341348    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 03:48:33.345091 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341351    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 03:48:33.345091 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341354    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 03:48:33.345091 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341357    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 03:48:33.345091 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341360    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 03:48:33.345091 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341363    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 03:48:33.345091 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341365    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 03:48:33.345091 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341368    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 03:48:33.345091 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341371    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 03:48:33.345091 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341375    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 03:48:33.345091 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.341381    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 03:48:33.345091 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341517    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 03:48:33.345593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341523    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 03:48:33.345593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341526    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 03:48:33.345593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341529    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 03:48:33.345593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341532    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 03:48:33.345593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341535    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 03:48:33.345593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341537    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 03:48:33.345593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341540    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 03:48:33.345593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341543    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 03:48:33.345593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341545    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 03:48:33.345593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341549    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 03:48:33.345593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341551    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 03:48:33.345593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341555    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 03:48:33.345593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341558    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 03:48:33.345593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341563    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 03:48:33.345593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341567    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 03:48:33.345593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341571    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 03:48:33.345593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341576    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 03:48:33.345593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341578    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 03:48:33.345593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341581    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 03:48:33.345593 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341583    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 03:48:33.346075 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341586    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 03:48:33.346075 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341588    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 03:48:33.346075 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341591    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 03:48:33.346075 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341595    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 03:48:33.346075 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341597    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 03:48:33.346075 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341600    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 03:48:33.346075 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341603    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 03:48:33.346075 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341605    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 03:48:33.346075 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341608    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 03:48:33.346075 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341611    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 03:48:33.346075 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341613    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 03:48:33.346075 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341615    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 03:48:33.346075 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341618    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 03:48:33.346075 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341621    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 03:48:33.346075 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341623    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 03:48:33.346075 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341626    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 03:48:33.346075 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341629    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 03:48:33.346075 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341632    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 03:48:33.346075 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341634    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 03:48:33.346075 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341637    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 03:48:33.346570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341640    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 03:48:33.346570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341644    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 03:48:33.346570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341649    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 03:48:33.346570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341653    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 03:48:33.346570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341658    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 03:48:33.346570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341663    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 03:48:33.346570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341666    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 03:48:33.346570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341669    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 03:48:33.346570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341672    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 03:48:33.346570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341675    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 03:48:33.346570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341678    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 03:48:33.346570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341681    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 03:48:33.346570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341684    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 03:48:33.346570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341687    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 03:48:33.346570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341689    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 03:48:33.346570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341692    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 03:48:33.346570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341695    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 03:48:33.346570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341698    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 03:48:33.346570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341701    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 03:48:33.346570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341704    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 03:48:33.347042 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341706    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 03:48:33.347042 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341709    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 03:48:33.347042 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341712    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 03:48:33.347042 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341714    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 03:48:33.347042 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341718    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 03:48:33.347042 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341722    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 03:48:33.347042 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341726    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 03:48:33.347042 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341731    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 03:48:33.347042 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341735    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 03:48:33.347042 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341738    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 03:48:33.347042 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341740    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 03:48:33.347042 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341743    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 03:48:33.347042 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341745    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 03:48:33.347042 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341748    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 03:48:33.347042 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341750    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 03:48:33.347042 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341754    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 03:48:33.347042 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341757    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 03:48:33.347042 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341761    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 03:48:33.347042 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341764    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 03:48:33.347502 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341768    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 03:48:33.347502 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341771    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 03:48:33.347502 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341775    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 03:48:33.347502 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341778    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 03:48:33.347502 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341781    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 03:48:33.347502 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341783    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 03:48:33.347502 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341786    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 03:48:33.347502 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341788    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 03:48:33.347502 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341791    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 03:48:33.347502 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341793    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 03:48:33.347502 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341796    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 03:48:33.347502 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341800    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 03:48:33.347502 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341804    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 03:48:33.347502 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341808    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 03:48:33.347502 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341812    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 03:48:33.347502 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341816    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 03:48:33.347502 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341819    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 03:48:33.347502 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341821    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 03:48:33.347502 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:33.341824    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 03:48:33.347976 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.341829    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 03:48:33.347976 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.342764    2583 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 07 03:48:33.347976 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.346744    2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 07 03:48:33.347976 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.347496    2583 server.go:1019] "Starting client certificate rotation"
Sep 07 03:48:33.347976 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.347595    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 07 03:48:33.347976 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.347626    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 07 03:48:33.350129 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.350114    2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 07 03:48:33.355051 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.355035    2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 07 03:48:33.362489 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.362466    2583 log.go:25] "Validated CRI v1 runtime API"
Sep 07 03:48:33.365413 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.365390    2583 log.go:25] "Validated CRI v1 image API"
Sep 07 03:48:33.366331 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.366312    2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 07 03:48:33.369140 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.369115    2583 fs.go:135] Filesystem UUIDs: map[250c7a61-89de-4b68-8fb4-3a220ce0c258:/dev/nvme0n1p3 53292fdc-10aa-4545-b561-1c9d02be16ac:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 07 03:48:33.369257 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.369139    2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 07 03:48:33.376169 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.376035    2583 manager.go:216] Machine: {Timestamp:2026-09-07 03:48:33.37485348 +0000 UTC m=+0.308334189 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099609 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c3282f877b705189972f484f4d7e9 SystemUUID:ec2c3282-f877-b705-1899-72f484f4d7e9 BootID:8ef172da-971e-45f6-a940-ff6b23364344 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0e:f9:5a:c4:d5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0e:f9:5a:c4:d5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:22:74:ff:8d:8d:d6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 07 03:48:33.376169 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.376147    2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 07 03:48:33.376374 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.376259    2583 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 07 03:48:33.377077 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.377048    2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 07 03:48:33.377235 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.377081    2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-175.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 07 03:48:33.377883 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.377872    2583 topology_manager.go:138] "Creating topology manager with none policy"
Sep 07 03:48:33.377916 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.377886    2583 container_manager_linux.go:309] "Creating device plugin manager"
Sep 07 03:48:33.377916 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.377894    2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 07 03:48:33.378195 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.378180    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 07 03:48:33.378412 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.378402    2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 07 03:48:33.379862 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.379851    2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 07 03:48:33.382987 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.382973    2583 state_mem.go:36] "Initialized new in-memory state store"
Sep 07 03:48:33.383110 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.383100    2583 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 07 03:48:33.383141 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.383130    2583 kubelet.go:486] "Attempting to sync node with API server"
Sep 07 03:48:33.383177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.383141    2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 07 03:48:33.383177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.383156    2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 07 03:48:33.383177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.383166    2583 kubelet.go:398] "Adding apiserver pod source"
Sep 07 03:48:33.383293 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.383179    2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 07 03:48:33.383556 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.383544    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 07 03:48:33.383612 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.383561    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 07 03:48:33.385560 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.385535    2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 07 03:48:33.388240 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.388204    2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 07 03:48:33.388329 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.388262    2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 07 03:48:33.388908 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.388894    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 07 03:48:33.388986 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.388917    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 07 03:48:33.388986 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.388926    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 07 03:48:33.388986 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.388934    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 07 03:48:33.388986 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.388947    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 07 03:48:33.388986 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.388955    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 07 03:48:33.388986 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.388975    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 07 03:48:33.388986 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.388983    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 07 03:48:33.389204 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.388994    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 07 03:48:33.389204 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.389004    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 07 03:48:33.389204 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.389024    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 07 03:48:33.389204 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.389033    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 07 03:48:33.389814 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.389802    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 07 03:48:33.389860 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.389818    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 07 03:48:33.391565 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:33.391539    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 07 03:48:33.391565 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.391558    2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-175.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 07 03:48:33.391715 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:33.391563    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-175.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 07 03:48:33.393697 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.393680    2583 server.go:1268] "Started kubelet"
Sep 07 03:48:33.393774 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.393731    2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 07 03:48:33.393956 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.393911    2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 07 03:48:33.394030 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.393980    2583 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 07 03:48:33.394686 ip-10-0-132-175 systemd[1]: Started Kubernetes Kubelet.
Sep 07 03:48:33.395522 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.395317    2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 07 03:48:33.396097 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.396090    2583 server.go:310] "Adding debug handlers to kubelet server"
Sep 07 03:48:33.398434 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:33.397741    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-175.ec2.internal.18d2ed4c29c6044b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-175.ec2.internal,UID:ip-10-0-132-175.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-175.ec2.internal,},FirstTimestamp:2026-09-07 03:48:33.393648715 +0000 UTC m=+0.327129485,LastTimestamp:2026-09-07 03:48:33.393648715 +0000 UTC m=+0.327129485,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-175.ec2.internal,}"
Sep 07 03:48:33.399897 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.399870    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jzff9"
Sep 07 03:48:33.400187 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.400171    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 07 03:48:33.400609 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:33.400591    2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 07 03:48:33.400833 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.400817    2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 07 03:48:33.400985 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:33.400950    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-175.ec2.internal\" not found"
Sep 07 03:48:33.401607 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.401570    2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 07 03:48:33.401607 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.401568    2583 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 07 03:48:33.401756 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.401614    2583 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 07 03:48:33.401756 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.401638    2583 factory.go:55] Registering systemd factory
Sep 07 03:48:33.401756 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.401653    2583 factory.go:223] Registration of the systemd container factory successfully
Sep 07 03:48:33.402034 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.402019    2583 reconstruct.go:97] "Volume reconstruction finished"
Sep 07 03:48:33.402034 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.402033    2583 reconciler.go:29] "Reconciler: start to sync state"
Sep 07 03:48:33.402263 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.402244    2583 factory.go:153] Registering CRI-O factory
Sep 07 03:48:33.402325 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.402268    2583 factory.go:223] Registration of the crio container factory successfully
Sep 07 03:48:33.402325 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.402322    2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 07 03:48:33.402413 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.402343    2583 factory.go:103] Registering Raw factory
Sep 07 03:48:33.402413 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.402359    2583 manager.go:1196] Started watching for new ooms in manager
Sep 07 03:48:33.402819 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.402806    2583 manager.go:349] Starting recovery of all containers
Sep 07 03:48:33.407756 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.407731    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jzff9"
Sep 07 03:48:33.410175 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:33.410133    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 07 03:48:33.410175 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:33.410125    2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-175.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 07 03:48:33.414400 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.414382    2583 manager.go:354] Recovery completed
Sep 07 03:48:33.415716 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:33.415692    2583 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 07 03:48:33.418899 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.418887    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 03:48:33.420897 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.420882    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-175.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 03:48:33.420960 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.420913    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-175.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 03:48:33.420960 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.420926    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-175.ec2.internal" event="NodeHasSufficientPID"
Sep 07 03:48:33.421577 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.421561    2583 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 07 03:48:33.421577 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.421576    2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 07 03:48:33.421683 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.421595    2583 state_mem.go:36] "Initialized new in-memory state store"
Sep 07 03:48:33.421683 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.421597    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 03:48:33.422388 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.422374    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-175.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 03:48:33.422438 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.422400    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-175.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 03:48:33.422438 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.422410    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-175.ec2.internal" event="NodeHasSufficientPID"
Sep 07 03:48:33.422737 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.422726    2583 policy_none.go:49] "None policy: Start"
Sep 07 03:48:33.422771 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.422745    2583 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 07 03:48:33.422771 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.422756    2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 07 03:48:33.423753 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.423741    2583 policy_none.go:47] "Start"
Sep 07 03:48:33.460910 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.460889    2583 manager.go:337] "Starting Device Plugin manager"
Sep 07 03:48:33.475040 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:33.460950    2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 07 03:48:33.475040 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.460961    2583 server.go:87] "Starting device plugin registration server"
Sep 07 03:48:33.475040 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.461342    2583 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 07 03:48:33.475040 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.461355    2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 07 03:48:33.475040 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.461456    2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 07 03:48:33.475040 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.461533    2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 07 03:48:33.475040 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.461540    2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 07 03:48:33.475040 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:33.462094    2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 07 03:48:33.475040 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:33.462150    2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-175.ec2.internal\" not found"
Sep 07 03:48:33.513473 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.513435    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 07 03:48:33.514953 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.514936    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 07 03:48:33.515053 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.514996    2583 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 07 03:48:33.515053 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.515026    2583 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 07 03:48:33.515141 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:33.515071    2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 07 03:48:33.517621 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.517596    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 03:48:33.561949 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.561912    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 03:48:33.563019 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.562994    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-175.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 03:48:33.563120 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.563031    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-175.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 03:48:33.563120 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.563042    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-175.ec2.internal" event="NodeHasSufficientPID"
Sep 07 03:48:33.563120 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.563067    2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-175.ec2.internal"
Sep 07 03:48:33.569773 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.569741    2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-175.ec2.internal"
Sep 07 03:48:33.569878 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:33.569776    2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-175.ec2.internal\": node \"ip-10-0-132-175.ec2.internal\" not found"
Sep 07 03:48:33.593021 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:33.592925    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-175.ec2.internal\" not found"
Sep 07 03:48:33.616132 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.616101    2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-175.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-175.ec2.internal"]
Sep 07 03:48:33.616277 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.616178    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 03:48:33.617851 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.617836    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-175.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 03:48:33.617926 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.617864    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-175.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 03:48:33.617926 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.617873    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-175.ec2.internal" event="NodeHasSufficientPID"
Sep 07 03:48:33.619069 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.619056    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 03:48:33.619199 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.619184    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-175.ec2.internal"
Sep 07 03:48:33.619275 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.619237    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 03:48:33.619820 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.619805    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-175.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 03:48:33.619888 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.619821    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-175.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 03:48:33.619888 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.619836    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-175.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 03:48:33.619888 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.619844    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-175.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 03:48:33.619888 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.619848    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-175.ec2.internal" event="NodeHasSufficientPID"
Sep 07 03:48:33.619888 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.619854    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-175.ec2.internal" event="NodeHasSufficientPID"
Sep 07 03:48:33.621135 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.621121    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-175.ec2.internal"
Sep 07 03:48:33.621190 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.621147    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 03:48:33.621995 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.621977    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-175.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 03:48:33.622053 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.622002    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-175.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 03:48:33.622053 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.622013    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-175.ec2.internal" event="NodeHasSufficientPID"
Sep 07 03:48:33.646978 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:33.646954    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-175.ec2.internal\" not found" node="ip-10-0-132-175.ec2.internal"
Sep 07 03:48:33.651334 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:33.651317    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-175.ec2.internal\" not found" node="ip-10-0-132-175.ec2.internal"
Sep 07 03:48:33.693907 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:33.693860    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-175.ec2.internal\" not found"
Sep 07 03:48:33.703548 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.703518    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0a39a5eb3a918e1ff8bcbe08348a2d2c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-175.ec2.internal\" (UID: \"0a39a5eb3a918e1ff8bcbe08348a2d2c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-175.ec2.internal"
Sep 07 03:48:33.703548 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.703547    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d405fbd5d2fd527a362872a57cd14648-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-175.ec2.internal\" (UID: \"d405fbd5d2fd527a362872a57cd14648\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-175.ec2.internal"
Sep 07 03:48:33.703731 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.703566    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d405fbd5d2fd527a362872a57cd14648-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-175.ec2.internal\" (UID: \"d405fbd5d2fd527a362872a57cd14648\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-175.ec2.internal"
Sep 07 03:48:33.794238 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:33.794177    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-175.ec2.internal\" not found"
Sep 07 03:48:33.804623 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.804604    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0a39a5eb3a918e1ff8bcbe08348a2d2c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-175.ec2.internal\" (UID: \"0a39a5eb3a918e1ff8bcbe08348a2d2c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-175.ec2.internal"
Sep 07 03:48:33.804717 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.804629    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d405fbd5d2fd527a362872a57cd14648-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-175.ec2.internal\" (UID: \"d405fbd5d2fd527a362872a57cd14648\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-175.ec2.internal"
Sep 07 03:48:33.804717 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.804646    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d405fbd5d2fd527a362872a57cd14648-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-175.ec2.internal\" (UID: \"d405fbd5d2fd527a362872a57cd14648\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-175.ec2.internal"
Sep 07 03:48:33.804808 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.804732    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d405fbd5d2fd527a362872a57cd14648-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-175.ec2.internal\" (UID: \"d405fbd5d2fd527a362872a57cd14648\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-175.ec2.internal"
Sep 07 03:48:33.804808 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.804739    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0a39a5eb3a918e1ff8bcbe08348a2d2c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-175.ec2.internal\" (UID: \"0a39a5eb3a918e1ff8bcbe08348a2d2c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-175.ec2.internal"
Sep 07 03:48:33.804808 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.804747    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d405fbd5d2fd527a362872a57cd14648-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-175.ec2.internal\" (UID: \"d405fbd5d2fd527a362872a57cd14648\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-175.ec2.internal"
Sep 07 03:48:33.895179 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:33.895097    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-175.ec2.internal\" not found"
Sep 07 03:48:33.949654 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.949615    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-175.ec2.internal"
Sep 07 03:48:33.954903 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:33.954886    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-175.ec2.internal"
Sep 07 03:48:33.995389 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:33.995351    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-175.ec2.internal\" not found"
Sep 07 03:48:34.095984 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:34.095935    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-175.ec2.internal\" not found"
Sep 07 03:48:34.196518 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:34.196439    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-175.ec2.internal\" not found"
Sep 07 03:48:34.297050 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:34.297013    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-175.ec2.internal\" not found"
Sep 07 03:48:34.347590 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:34.347560    2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 07 03:48:34.348021 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:34.347712    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 07 03:48:34.397241 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:34.397197    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-175.ec2.internal\" not found"
Sep 07 03:48:34.410044 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:34.410011    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-06 03:43:34 +0000 UTC" deadline="2028-03-18 20:46:01.768115134 +0000 UTC"
Sep 07 03:48:34.410044 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:34.410040    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13408h57m27.358079391s"
Sep 07 03:48:34.490702 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:34.490619    2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 03:48:34.497477 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:34.497455    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-175.ec2.internal\" not found"
Sep 07 03:48:34.501379 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:34.501349    2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 03:48:34.538028 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:34.537999    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd405fbd5d2fd527a362872a57cd14648.slice/crio-d07e7260439b46d52a847f687276dbe088df323403b40f9a67a54c59db711847 WatchSource:0}: Error finding container d07e7260439b46d52a847f687276dbe088df323403b40f9a67a54c59db711847: Status 404 returned error can't find the container with id d07e7260439b46d52a847f687276dbe088df323403b40f9a67a54c59db711847
Sep 07 03:48:34.538986 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:34.538960    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a39a5eb3a918e1ff8bcbe08348a2d2c.slice/crio-0c4ac71e2a3a5e339b0005abddc73f012f55f11934c669afd749fec3cc9ba5d4 WatchSource:0}: Error finding container 0c4ac71e2a3a5e339b0005abddc73f012f55f11934c669afd749fec3cc9ba5d4: Status 404 returned error can't find the container with id 0c4ac71e2a3a5e339b0005abddc73f012f55f11934c669afd749fec3cc9ba5d4
Sep 07 03:48:34.543221 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:34.543189    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 03:48:34.601596 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:34.601569    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-175.ec2.internal"
Sep 07 03:48:34.610787 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:34.610761    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 07 03:48:34.612285 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:34.612271    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-175.ec2.internal"
Sep 07 03:48:34.622994 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:34.622979    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 07 03:48:34.720486 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:34.720457    2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 03:48:35.384332 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.384208    2583 apiserver.go:52] "Watching apiserver"
Sep 07 03:48:35.392232 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.392200    2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 07 03:48:35.394289 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.394256    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-xg5w5","openshift-network-operator/iptables-alerter-8mbdb","openshift-ovn-kubernetes/ovnkube-node-nk8kh","kube-system/global-pull-secret-syncer-v9fzl","kube-system/konnectivity-agent-gw9hg","kube-system/kube-apiserver-proxy-ip-10-0-132-175.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-175.ec2.internal","openshift-multus/multus-9cztj","openshift-multus/multus-additional-cni-plugins-prd5g","openshift-multus/network-metrics-daemon-sml8b","openshift-network-diagnostics/network-check-target-qbsbx","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m","openshift-cluster-node-tuning-operator/tuned-jzp9s","openshift-dns/node-resolver-px55t"]
Sep 07 03:48:35.397422 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.397399    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:48:35.397550 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:35.397493    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sml8b" podUID="bb13bb24-ca38-4d13-aa0c-1c47eb028ae1"
Sep 07 03:48:35.399724 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.399695    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.401644 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.401623    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 07 03:48:35.402067 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.402046    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.402901 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.402875    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8jhzj\""
Sep 07 03:48:35.403009 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.402913    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 07 03:48:35.403009 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.402970    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 07 03:48:35.404305 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.404267    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 07 03:48:35.404651 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.404617    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 07 03:48:35.405630 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.405607    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 07 03:48:35.405948 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.405928    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:35.406036 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.405965    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 07 03:48:35.406036 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:35.406016    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v9fzl" podUID="244e7a50-68b4-41dc-aa4b-8b83813cf489"
Sep 07 03:48:35.406382 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.406362    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 07 03:48:35.406866 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.406846    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 07 03:48:35.407387 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.407369    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-t2p9j\""
Sep 07 03:48:35.409680 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.409659    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gw9hg"
Sep 07 03:48:35.412409 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.412387    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-px55t"
Sep 07 03:48:35.412850 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.412825    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-run-systemd\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.412966 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.412862    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-sys\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.412966 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.412889    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-host-slash\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.412966 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.412915    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr6ql\" (UniqueName: \"kubernetes.io/projected/de89531d-86b1-404c-8140-2ffa057b333b-kube-api-access-vr6ql\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.413149 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.412963    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs\") pod \"network-metrics-daemon-sml8b\" (UID: \"bb13bb24-ca38-4d13-aa0c-1c47eb028ae1\") " pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:48:35.413149 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413019    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-etc-systemd\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.413149 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413035    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9hhbp\""
Sep 07 03:48:35.413149 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413054    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-systemd-units\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.413149 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413100    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/de89531d-86b1-404c-8140-2ffa057b333b-env-overrides\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.413149 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413124    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-etc-sysctl-d\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.413455 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413150    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-etc-sysctl-conf\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.413455 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413165    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 07 03:48:35.413455 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413034    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 07 03:48:35.413455 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413266    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-var-lib-kubelet\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.413455 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413306    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-host-run-netns\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.413455 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413352    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-node-log\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.413455 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413385    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/de89531d-86b1-404c-8140-2ffa057b333b-ovn-node-metrics-cert\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.413455 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413428    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjm2c\" (UniqueName: \"kubernetes.io/projected/af1e9edf-b324-494e-aa39-b1351e2dd686-kube-api-access-jjm2c\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.413919 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413465    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-var-lib-openvswitch\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.413919 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413491    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-run-ovn\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.413919 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413543    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-host-run-ovn-kubernetes\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.413919 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413624    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-host-cni-bin\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.413919 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413644    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.413919 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413668    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-run\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.413919 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413701    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-host\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.413919 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413718    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-host-kubelet\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.413919 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413736    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/de89531d-86b1-404c-8140-2ffa057b333b-ovnkube-script-lib\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.413919 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413762    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4g6w\" (UniqueName: \"kubernetes.io/projected/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-kube-api-access-g4g6w\") pod \"network-metrics-daemon-sml8b\" (UID: \"bb13bb24-ca38-4d13-aa0c-1c47eb028ae1\") " pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:48:35.413919 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413786    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-etc-sysconfig\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.413919 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413804    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-etc-kubernetes\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.413919 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413818    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/af1e9edf-b324-494e-aa39-b1351e2dd686-etc-tuned\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.413919 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413836    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af1e9edf-b324-494e-aa39-b1351e2dd686-tmp\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.413919 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413860    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-etc-openvswitch\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.413919 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413912    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-etc-modprobe-d\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.414883 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413937    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-lib-modules\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.414883 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.413973    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-run-openvswitch\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.414883 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.414003    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-log-socket\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.414883 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.414031    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-host-cni-netd\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.414883 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.414060    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/de89531d-86b1-404c-8140-2ffa057b333b-ovnkube-config\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.414883 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.414752    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ts8xw\""
Sep 07 03:48:35.414883 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.414878    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 07 03:48:35.415156 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.415101    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 07 03:48:35.415209 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.415198    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-prd5g"
Sep 07 03:48:35.416672 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.416649    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 07 03:48:35.418177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.417468    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 07 03:48:35.418177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.417816    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 07 03:48:35.418177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.417841    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 07 03:48:35.418177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.418069    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 07 03:48:35.418177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.418108    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 07 03:48:35.418535 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.418377    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-25jrp\""
Sep 07 03:48:35.418816 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.418789    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.420866 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.420848    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 07 03:48:35.421388 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.421370    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qzxbr\""
Sep 07 03:48:35.421711 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.421696    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8mbdb"
Sep 07 03:48:35.423990 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.423922    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 07 03:48:35.424296 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.424146    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 07 03:48:35.424296 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.424228    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 07 03:48:35.424296 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.424263    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xg5w5"
Sep 07 03:48:35.424521 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.424357    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qqvgn\""
Sep 07 03:48:35.426668 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.426647    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dwtc8\""
Sep 07 03:48:35.426668 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.426659    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 07 03:48:35.426815 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.426649    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 07 03:48:35.426815 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.426746    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 07 03:48:35.426971 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.426957    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:48:35.427063 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:35.427039    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbsbx" podUID="4cf74d00-0e92-415e-8129-2033297d3707"
Sep 07 03:48:35.429709 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.429688    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m"
Sep 07 03:48:35.431955 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.431937    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 07 03:48:35.432049 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.432000    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 07 03:48:35.432254 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.432181    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 07 03:48:35.432353 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.432309    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-j2hw7\""
Sep 07 03:48:35.438999 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.438980    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-frprb"
Sep 07 03:48:35.448347 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.448323    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-frprb"
Sep 07 03:48:35.502845 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.502811    2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 07 03:48:35.514870 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.514829    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-run-systemd\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.515068 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.514880    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-sys\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.515068 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.514920    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh8wl\" (UniqueName: \"kubernetes.io/projected/fa0c971c-a802-448e-a954-eaa78c38cb79-kube-api-access-sh8wl\") pod \"aws-ebs-csi-driver-node-mbs8m\" (UID: \"fa0c971c-a802-448e-a954-eaa78c38cb79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m"
Sep 07 03:48:35.515068 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.514949    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-host-slash\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.515068 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.514977    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr6ql\" (UniqueName: \"kubernetes.io/projected/de89531d-86b1-404c-8140-2ffa057b333b-kube-api-access-vr6ql\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.515068 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.514979    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-run-systemd\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.515068 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515003    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-etc-systemd\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.515068 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515032    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret\") pod \"global-pull-secret-syncer-v9fzl\" (UID: \"244e7a50-68b4-41dc-aa4b-8b83813cf489\") " pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:35.515068 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515064    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-sys\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.515415 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515062    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/de17335a-7920-400b-ad6d-73fe656c84e0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-prd5g\" (UID: \"de17335a-7920-400b-ad6d-73fe656c84e0\") " pod="openshift-multus/multus-additional-cni-plugins-prd5g"
Sep 07 03:48:35.515415 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515113    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fa0c971c-a802-448e-a954-eaa78c38cb79-device-dir\") pod \"aws-ebs-csi-driver-node-mbs8m\" (UID: \"fa0c971c-a802-448e-a954-eaa78c38cb79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m"
Sep 07 03:48:35.515415 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515128    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-host-slash\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.515415 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515151    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-systemd-units\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.515415 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515181    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-host-run-multus-certs\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.515415 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515207    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9ae1bb54-3861-4759-9041-c97a6bf9b6b5-host-slash\") pod \"iptables-alerter-8mbdb\" (UID: \"9ae1bb54-3861-4759-9041-c97a6bf9b6b5\") " pod="openshift-network-operator/iptables-alerter-8mbdb"
Sep 07 03:48:35.515415 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515229    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-host-run-netns\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.515415 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515270    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-node-log\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.515415 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515293    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjm2c\" (UniqueName: \"kubernetes.io/projected/af1e9edf-b324-494e-aa39-b1351e2dd686-kube-api-access-jjm2c\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.515415 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515309    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-host-run-netns\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.515415 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515324    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-run-ovn\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.515415 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515341    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.515415 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515358    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-host\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.515415 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515382    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af1e9edf-b324-494e-aa39-b1351e2dd686-tmp\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.515415 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515406    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/503f1518-10bb-424b-87b0-00bf7cf2239f-hosts-file\") pod \"node-resolver-px55t\" (UID: \"503f1518-10bb-424b-87b0-00bf7cf2239f\") " pod="openshift-dns/node-resolver-px55t"
Sep 07 03:48:35.515960 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515431    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f5e84b44-6d3b-464b-a987-fff4a928f773-agent-certs\") pod \"konnectivity-agent-gw9hg\" (UID: \"f5e84b44-6d3b-464b-a987-fff4a928f773\") " pod="kube-system/konnectivity-agent-gw9hg"
Sep 07 03:48:35.515960 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515458    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52683f85-6093-473d-9e11-541e6ec8b2fb-host\") pod \"node-ca-xg5w5\" (UID: \"52683f85-6093-473d-9e11-541e6ec8b2fb\") " pod="openshift-image-registry/node-ca-xg5w5"
Sep 07 03:48:35.515960 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515474    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-host-kubelet\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.515960 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515490    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4g6w\" (UniqueName: \"kubernetes.io/projected/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-kube-api-access-g4g6w\") pod \"network-metrics-daemon-sml8b\" (UID: \"bb13bb24-ca38-4d13-aa0c-1c47eb028ae1\") " pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:48:35.515960 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515505    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/af1e9edf-b324-494e-aa39-b1351e2dd686-etc-tuned\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.515960 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515507    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-etc-systemd\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.515960 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515521    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfcn7\" (UniqueName: \"kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7\") pod \"network-check-target-qbsbx\" (UID: \"4cf74d00-0e92-415e-8129-2033297d3707\") " pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:48:35.515960 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515539    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/de17335a-7920-400b-ad6d-73fe656c84e0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-prd5g\" (UID: \"de17335a-7920-400b-ad6d-73fe656c84e0\") " pod="openshift-multus/multus-additional-cni-plugins-prd5g"
Sep 07 03:48:35.515960 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515555    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-host-var-lib-kubelet\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.515960 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515571    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-multus-conf-dir\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.515960 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515573    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.515960 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515586    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8c1667f2-8f99-44ed-96de-544fea7fe5a6-multus-daemon-config\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.515960 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515617    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgwcp\" (UniqueName: \"kubernetes.io/projected/8c1667f2-8f99-44ed-96de-544fea7fe5a6-kube-api-access-cgwcp\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.515960 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515714    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-run-ovn\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.515960 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515761    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-host\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.515960 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515794    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-run-openvswitch\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.515960 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515820    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-log-socket\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.516851 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515828    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-systemd-units\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.516851 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515846    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/de89531d-86b1-404c-8140-2ffa057b333b-ovnkube-config\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.516851 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515874    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/244e7a50-68b4-41dc-aa4b-8b83813cf489-dbus\") pod \"global-pull-secret-syncer-v9fzl\" (UID: \"244e7a50-68b4-41dc-aa4b-8b83813cf489\") " pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:35.516851 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515902    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsd4p\" (UniqueName: \"kubernetes.io/projected/503f1518-10bb-424b-87b0-00bf7cf2239f-kube-api-access-vsd4p\") pod \"node-resolver-px55t\" (UID: \"503f1518-10bb-424b-87b0-00bf7cf2239f\") " pod="openshift-dns/node-resolver-px55t"
Sep 07 03:48:35.516851 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515930    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-host-var-lib-cni-multus\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.516851 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515957    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fa0c971c-a802-448e-a954-eaa78c38cb79-etc-selinux\") pod \"aws-ebs-csi-driver-node-mbs8m\" (UID: \"fa0c971c-a802-448e-a954-eaa78c38cb79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m"
Sep 07 03:48:35.516851 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.515984    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs\") pod \"network-metrics-daemon-sml8b\" (UID: \"bb13bb24-ca38-4d13-aa0c-1c47eb028ae1\") " pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:48:35.516851 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516037    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-system-cni-dir\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.516851 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516072    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-multus-socket-dir-parent\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.516851 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:35.516082    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 03:48:35.516851 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516100    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/52683f85-6093-473d-9e11-541e6ec8b2fb-serviceca\") pod \"node-ca-xg5w5\" (UID: \"52683f85-6093-473d-9e11-541e6ec8b2fb\") " pod="openshift-image-registry/node-ca-xg5w5"
Sep 07 03:48:35.516851 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:35.516168    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs podName:bb13bb24-ca38-4d13-aa0c-1c47eb028ae1 nodeName:}" failed. No retries permitted until 2026-09-07 03:48:36.01612873 +0000 UTC m=+2.949609435 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs") pod "network-metrics-daemon-sml8b" (UID: "bb13bb24-ca38-4d13-aa0c-1c47eb028ae1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 03:48:35.516851 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516382    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-run-openvswitch\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.516851 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516401    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fa0c971c-a802-448e-a954-eaa78c38cb79-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mbs8m\" (UID: \"fa0c971c-a802-448e-a954-eaa78c38cb79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m"
Sep 07 03:48:35.516851 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516408    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-log-socket\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.516851 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516418    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-host-run-netns\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.516851 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516264    2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 07 03:48:35.517585 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516439    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fa0c971c-a802-448e-a954-eaa78c38cb79-sys-fs\") pod \"aws-ebs-csi-driver-node-mbs8m\" (UID: \"fa0c971c-a802-448e-a954-eaa78c38cb79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m"
Sep 07 03:48:35.517585 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516467    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/de89531d-86b1-404c-8140-2ffa057b333b-env-overrides\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.517585 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516495    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-etc-sysctl-d\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.517585 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516560    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-etc-sysctl-conf\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.517585 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516591    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-var-lib-kubelet\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.517585 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516622    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/244e7a50-68b4-41dc-aa4b-8b83813cf489-kubelet-config\") pod \"global-pull-secret-syncer-v9fzl\" (UID: \"244e7a50-68b4-41dc-aa4b-8b83813cf489\") " pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:35.517585 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516652    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-host-var-lib-cni-bin\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.517585 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516681    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx8qm\" (UniqueName: \"kubernetes.io/projected/9ae1bb54-3861-4759-9041-c97a6bf9b6b5-kube-api-access-wx8qm\") pod \"iptables-alerter-8mbdb\" (UID: \"9ae1bb54-3861-4759-9041-c97a6bf9b6b5\") " pod="openshift-network-operator/iptables-alerter-8mbdb"
Sep 07 03:48:35.517585 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516713    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/de89531d-86b1-404c-8140-2ffa057b333b-ovn-node-metrics-cert\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.517585 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516739    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/de17335a-7920-400b-ad6d-73fe656c84e0-cni-binary-copy\") pod \"multus-additional-cni-plugins-prd5g\" (UID: \"de17335a-7920-400b-ad6d-73fe656c84e0\") " pod="openshift-multus/multus-additional-cni-plugins-prd5g"
Sep 07 03:48:35.517585 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516765    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/de17335a-7920-400b-ad6d-73fe656c84e0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-prd5g\" (UID: \"de17335a-7920-400b-ad6d-73fe656c84e0\") " pod="openshift-multus/multus-additional-cni-plugins-prd5g"
Sep 07 03:48:35.517585 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516783    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n9x9\" (UniqueName: \"kubernetes.io/projected/de17335a-7920-400b-ad6d-73fe656c84e0-kube-api-access-4n9x9\") pod \"multus-additional-cni-plugins-prd5g\" (UID: \"de17335a-7920-400b-ad6d-73fe656c84e0\") " pod="openshift-multus/multus-additional-cni-plugins-prd5g"
Sep 07 03:48:35.517585 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516800    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-etc-kubernetes\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.517585 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516887    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-multus-cni-dir\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.517585 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516919    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-var-lib-openvswitch\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.517585 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516952    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-host-run-ovn-kubernetes\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.518159 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516969    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-host-cni-bin\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.518159 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.516986    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-run\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.518159 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517003    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f5e84b44-6d3b-464b-a987-fff4a928f773-konnectivity-ca\") pod \"konnectivity-agent-gw9hg\" (UID: \"f5e84b44-6d3b-464b-a987-fff4a928f773\") " pod="kube-system/konnectivity-agent-gw9hg"
Sep 07 03:48:35.518159 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517027    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fa0c971c-a802-448e-a954-eaa78c38cb79-socket-dir\") pod \"aws-ebs-csi-driver-node-mbs8m\" (UID: \"fa0c971c-a802-448e-a954-eaa78c38cb79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m"
Sep 07 03:48:35.518159 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517062    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/de89531d-86b1-404c-8140-2ffa057b333b-ovnkube-script-lib\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.518159 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517082    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-etc-sysconfig\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.518159 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517108    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-etc-kubernetes\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.518159 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517135    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/503f1518-10bb-424b-87b0-00bf7cf2239f-tmp-dir\") pod \"node-resolver-px55t\" (UID: \"503f1518-10bb-424b-87b0-00bf7cf2239f\") " pod="openshift-dns/node-resolver-px55t"
Sep 07 03:48:35.518159 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517160    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/de17335a-7920-400b-ad6d-73fe656c84e0-cnibin\") pod \"multus-additional-cni-plugins-prd5g\" (UID: \"de17335a-7920-400b-ad6d-73fe656c84e0\") " pod="openshift-multus/multus-additional-cni-plugins-prd5g"
Sep 07 03:48:35.518159 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517182    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-cnibin\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.518159 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517203    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-host-run-k8s-cni-cncf-io\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.518159 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517220    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9ae1bb54-3861-4759-9041-c97a6bf9b6b5-iptables-alerter-script\") pod \"iptables-alerter-8mbdb\" (UID: \"9ae1bb54-3861-4759-9041-c97a6bf9b6b5\") " pod="openshift-network-operator/iptables-alerter-8mbdb"
Sep 07 03:48:35.518159 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517245    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8c1667f2-8f99-44ed-96de-544fea7fe5a6-cni-binary-copy\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.518159 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517269    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-etc-openvswitch\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.518159 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517305    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-etc-modprobe-d\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.518159 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517330    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-lib-modules\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.518159 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517327    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/de89531d-86b1-404c-8140-2ffa057b333b-ovnkube-config\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.518948 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517437    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de17335a-7920-400b-ad6d-73fe656c84e0-system-cni-dir\") pod \"multus-additional-cni-plugins-prd5g\" (UID: \"de17335a-7920-400b-ad6d-73fe656c84e0\") " pod="openshift-multus/multus-additional-cni-plugins-prd5g"
Sep 07 03:48:35.518948 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517467    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5jpz\" (UniqueName: \"kubernetes.io/projected/52683f85-6093-473d-9e11-541e6ec8b2fb-kube-api-access-d5jpz\") pod \"node-ca-xg5w5\" (UID: \"52683f85-6093-473d-9e11-541e6ec8b2fb\") " pod="openshift-image-registry/node-ca-xg5w5"
Sep 07 03:48:35.518948 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517492    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fa0c971c-a802-448e-a954-eaa78c38cb79-registration-dir\") pod \"aws-ebs-csi-driver-node-mbs8m\" (UID: \"fa0c971c-a802-448e-a954-eaa78c38cb79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m"
Sep 07 03:48:35.518948 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517518    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-host-cni-netd\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.518948 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517541    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-hostroot\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.518948 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517546    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-lib-modules\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.518948 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517606    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-var-lib-openvswitch\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.518948 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517637    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-host-run-ovn-kubernetes\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.518948 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517643    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-etc-sysctl-d\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.518948 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517664    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-host-cni-bin\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.518948 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517687    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-node-log\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.518948 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517760    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-run\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.518948 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.517919    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-host-kubelet\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.518948 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.518231    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/de89531d-86b1-404c-8140-2ffa057b333b-ovnkube-script-lib\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.518948 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.518319    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-etc-sysconfig\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.518948 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.518364    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-etc-openvswitch\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.518948 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.518443    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-etc-kubernetes\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.519842 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.518471    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/de89531d-86b1-404c-8140-2ffa057b333b-env-overrides\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.519842 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.518508    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-etc-modprobe-d\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.519842 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.518620    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-etc-sysctl-conf\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.519842 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.518664    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/de89531d-86b1-404c-8140-2ffa057b333b-host-cni-netd\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.519842 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.518712    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af1e9edf-b324-494e-aa39-b1351e2dd686-var-lib-kubelet\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.521120 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.521099    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/af1e9edf-b324-494e-aa39-b1351e2dd686-etc-tuned\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.521530 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.521491    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af1e9edf-b324-494e-aa39-b1351e2dd686-tmp\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.521814 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.521748    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/de89531d-86b1-404c-8140-2ffa057b333b-ovn-node-metrics-cert\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.523336 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.523279    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-175.ec2.internal" event={"ID":"d405fbd5d2fd527a362872a57cd14648","Type":"ContainerStarted","Data":"d07e7260439b46d52a847f687276dbe088df323403b40f9a67a54c59db711847"}
Sep 07 03:48:35.525682 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.525651    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-175.ec2.internal" event={"ID":"0a39a5eb3a918e1ff8bcbe08348a2d2c","Type":"ContainerStarted","Data":"0c4ac71e2a3a5e339b0005abddc73f012f55f11934c669afd749fec3cc9ba5d4"}
Sep 07 03:48:35.526303 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.526266    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr6ql\" (UniqueName: \"kubernetes.io/projected/de89531d-86b1-404c-8140-2ffa057b333b-kube-api-access-vr6ql\") pod \"ovnkube-node-nk8kh\" (UID: \"de89531d-86b1-404c-8140-2ffa057b333b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.527002 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.526980    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjm2c\" (UniqueName: \"kubernetes.io/projected/af1e9edf-b324-494e-aa39-b1351e2dd686-kube-api-access-jjm2c\") pod \"tuned-jzp9s\" (UID: \"af1e9edf-b324-494e-aa39-b1351e2dd686\") " pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.528171 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.528147    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4g6w\" (UniqueName: \"kubernetes.io/projected/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-kube-api-access-g4g6w\") pod \"network-metrics-daemon-sml8b\" (UID: \"bb13bb24-ca38-4d13-aa0c-1c47eb028ae1\") " pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:48:35.618431 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.618389    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-host-run-multus-certs\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.618431 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.618438    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9ae1bb54-3861-4759-9041-c97a6bf9b6b5-host-slash\") pod \"iptables-alerter-8mbdb\" (UID: \"9ae1bb54-3861-4759-9041-c97a6bf9b6b5\") " pod="openshift-network-operator/iptables-alerter-8mbdb"
Sep 07 03:48:35.618712 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.618466    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-host-run-netns\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.618712 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.618497    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/503f1518-10bb-424b-87b0-00bf7cf2239f-hosts-file\") pod \"node-resolver-px55t\" (UID: \"503f1518-10bb-424b-87b0-00bf7cf2239f\") " pod="openshift-dns/node-resolver-px55t"
Sep 07 03:48:35.618712 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.618545    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f5e84b44-6d3b-464b-a987-fff4a928f773-agent-certs\") pod \"konnectivity-agent-gw9hg\" (UID: \"f5e84b44-6d3b-464b-a987-fff4a928f773\") " pod="kube-system/konnectivity-agent-gw9hg"
Sep 07 03:48:35.618712 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.618573    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52683f85-6093-473d-9e11-541e6ec8b2fb-host\") pod \"node-ca-xg5w5\" (UID: \"52683f85-6093-473d-9e11-541e6ec8b2fb\") " pod="openshift-image-registry/node-ca-xg5w5"
Sep 07 03:48:35.618712 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.618596    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfcn7\" (UniqueName: \"kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7\") pod \"network-check-target-qbsbx\" (UID: \"4cf74d00-0e92-415e-8129-2033297d3707\") " pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:48:35.618712 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.618624    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/de17335a-7920-400b-ad6d-73fe656c84e0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-prd5g\" (UID: \"de17335a-7920-400b-ad6d-73fe656c84e0\") " pod="openshift-multus/multus-additional-cni-plugins-prd5g"
Sep 07 03:48:35.618712 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.618651    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-host-var-lib-kubelet\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.618712 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.618678    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-multus-conf-dir\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.618712 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.618697    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8c1667f2-8f99-44ed-96de-544fea7fe5a6-multus-daemon-config\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.618712 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.618715    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgwcp\" (UniqueName: \"kubernetes.io/projected/8c1667f2-8f99-44ed-96de-544fea7fe5a6-kube-api-access-cgwcp\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.619177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.618741    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/244e7a50-68b4-41dc-aa4b-8b83813cf489-dbus\") pod \"global-pull-secret-syncer-v9fzl\" (UID: \"244e7a50-68b4-41dc-aa4b-8b83813cf489\") " pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:35.619177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.618767    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsd4p\" (UniqueName: \"kubernetes.io/projected/503f1518-10bb-424b-87b0-00bf7cf2239f-kube-api-access-vsd4p\") pod \"node-resolver-px55t\" (UID: \"503f1518-10bb-424b-87b0-00bf7cf2239f\") " pod="openshift-dns/node-resolver-px55t"
Sep 07 03:48:35.619177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.618793    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-host-var-lib-cni-multus\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.619177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.618821    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fa0c971c-a802-448e-a954-eaa78c38cb79-etc-selinux\") pod \"aws-ebs-csi-driver-node-mbs8m\" (UID: \"fa0c971c-a802-448e-a954-eaa78c38cb79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m"
Sep 07 03:48:35.619177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.618854    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-system-cni-dir\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.619177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.618880    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-multus-socket-dir-parent\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.619177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.618903    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/52683f85-6093-473d-9e11-541e6ec8b2fb-serviceca\") pod \"node-ca-xg5w5\" (UID: \"52683f85-6093-473d-9e11-541e6ec8b2fb\") " pod="openshift-image-registry/node-ca-xg5w5"
Sep 07 03:48:35.619177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.618931    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fa0c971c-a802-448e-a954-eaa78c38cb79-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mbs8m\" (UID: \"fa0c971c-a802-448e-a954-eaa78c38cb79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m"
Sep 07 03:48:35.619177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.618951    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fa0c971c-a802-448e-a954-eaa78c38cb79-sys-fs\") pod \"aws-ebs-csi-driver-node-mbs8m\" (UID: \"fa0c971c-a802-448e-a954-eaa78c38cb79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m"
Sep 07 03:48:35.619177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.618981    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/244e7a50-68b4-41dc-aa4b-8b83813cf489-kubelet-config\") pod \"global-pull-secret-syncer-v9fzl\" (UID: \"244e7a50-68b4-41dc-aa4b-8b83813cf489\") " pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:35.619177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619003    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-host-var-lib-cni-bin\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.619177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619020    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx8qm\" (UniqueName: \"kubernetes.io/projected/9ae1bb54-3861-4759-9041-c97a6bf9b6b5-kube-api-access-wx8qm\") pod \"iptables-alerter-8mbdb\" (UID: \"9ae1bb54-3861-4759-9041-c97a6bf9b6b5\") " pod="openshift-network-operator/iptables-alerter-8mbdb"
Sep 07 03:48:35.619177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619044    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/de17335a-7920-400b-ad6d-73fe656c84e0-cni-binary-copy\") pod \"multus-additional-cni-plugins-prd5g\" (UID: \"de17335a-7920-400b-ad6d-73fe656c84e0\") " pod="openshift-multus/multus-additional-cni-plugins-prd5g"
Sep 07 03:48:35.619177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619065    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/de17335a-7920-400b-ad6d-73fe656c84e0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-prd5g\" (UID: \"de17335a-7920-400b-ad6d-73fe656c84e0\") " pod="openshift-multus/multus-additional-cni-plugins-prd5g"
Sep 07 03:48:35.619177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619084    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n9x9\" (UniqueName: \"kubernetes.io/projected/de17335a-7920-400b-ad6d-73fe656c84e0-kube-api-access-4n9x9\") pod \"multus-additional-cni-plugins-prd5g\" (UID: \"de17335a-7920-400b-ad6d-73fe656c84e0\") " pod="openshift-multus/multus-additional-cni-plugins-prd5g"
Sep 07 03:48:35.619177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619113    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-etc-kubernetes\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.619177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619132    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-multus-cni-dir\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.620002 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619162    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f5e84b44-6d3b-464b-a987-fff4a928f773-konnectivity-ca\") pod \"konnectivity-agent-gw9hg\" (UID: \"f5e84b44-6d3b-464b-a987-fff4a928f773\") " pod="kube-system/konnectivity-agent-gw9hg"
Sep 07 03:48:35.620002 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619185    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fa0c971c-a802-448e-a954-eaa78c38cb79-socket-dir\") pod \"aws-ebs-csi-driver-node-mbs8m\" (UID: \"fa0c971c-a802-448e-a954-eaa78c38cb79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m"
Sep 07 03:48:35.620002 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619211    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/503f1518-10bb-424b-87b0-00bf7cf2239f-tmp-dir\") pod \"node-resolver-px55t\" (UID: \"503f1518-10bb-424b-87b0-00bf7cf2239f\") " pod="openshift-dns/node-resolver-px55t"
Sep 07 03:48:35.620002 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619236    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/de17335a-7920-400b-ad6d-73fe656c84e0-cnibin\") pod \"multus-additional-cni-plugins-prd5g\" (UID: \"de17335a-7920-400b-ad6d-73fe656c84e0\") " pod="openshift-multus/multus-additional-cni-plugins-prd5g"
Sep 07 03:48:35.620002 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619255    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-cnibin\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.620002 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619280    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-host-run-k8s-cni-cncf-io\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.620002 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619327    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9ae1bb54-3861-4759-9041-c97a6bf9b6b5-iptables-alerter-script\") pod \"iptables-alerter-8mbdb\" (UID: \"9ae1bb54-3861-4759-9041-c97a6bf9b6b5\") " pod="openshift-network-operator/iptables-alerter-8mbdb"
Sep 07 03:48:35.620002 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619355    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8c1667f2-8f99-44ed-96de-544fea7fe5a6-cni-binary-copy\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.620002 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619387    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de17335a-7920-400b-ad6d-73fe656c84e0-system-cni-dir\") pod \"multus-additional-cni-plugins-prd5g\" (UID: \"de17335a-7920-400b-ad6d-73fe656c84e0\") " pod="openshift-multus/multus-additional-cni-plugins-prd5g"
Sep 07 03:48:35.620002 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619412    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5jpz\" (UniqueName: \"kubernetes.io/projected/52683f85-6093-473d-9e11-541e6ec8b2fb-kube-api-access-d5jpz\") pod \"node-ca-xg5w5\" (UID: \"52683f85-6093-473d-9e11-541e6ec8b2fb\") " pod="openshift-image-registry/node-ca-xg5w5"
Sep 07 03:48:35.620002 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619438    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fa0c971c-a802-448e-a954-eaa78c38cb79-registration-dir\") pod \"aws-ebs-csi-driver-node-mbs8m\" (UID: \"fa0c971c-a802-448e-a954-eaa78c38cb79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m"
Sep 07 03:48:35.620002 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619463    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-hostroot\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.620002 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619498    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh8wl\" (UniqueName: \"kubernetes.io/projected/fa0c971c-a802-448e-a954-eaa78c38cb79-kube-api-access-sh8wl\") pod \"aws-ebs-csi-driver-node-mbs8m\" (UID: \"fa0c971c-a802-448e-a954-eaa78c38cb79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m"
Sep 07 03:48:35.620002 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619531    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret\") pod \"global-pull-secret-syncer-v9fzl\" (UID: \"244e7a50-68b4-41dc-aa4b-8b83813cf489\") " pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:35.620002 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619553    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/de17335a-7920-400b-ad6d-73fe656c84e0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-prd5g\" (UID: \"de17335a-7920-400b-ad6d-73fe656c84e0\") " pod="openshift-multus/multus-additional-cni-plugins-prd5g"
Sep 07 03:48:35.620002 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619573    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fa0c971c-a802-448e-a954-eaa78c38cb79-device-dir\") pod \"aws-ebs-csi-driver-node-mbs8m\" (UID: \"fa0c971c-a802-448e-a954-eaa78c38cb79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m"
Sep 07 03:48:35.620002 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619685    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fa0c971c-a802-448e-a954-eaa78c38cb79-device-dir\") pod \"aws-ebs-csi-driver-node-mbs8m\" (UID: \"fa0c971c-a802-448e-a954-eaa78c38cb79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m"
Sep 07 03:48:35.620856 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619741    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-host-run-multus-certs\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.620856 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619783    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9ae1bb54-3861-4759-9041-c97a6bf9b6b5-host-slash\") pod \"iptables-alerter-8mbdb\" (UID: \"9ae1bb54-3861-4759-9041-c97a6bf9b6b5\") " pod="openshift-network-operator/iptables-alerter-8mbdb"
Sep 07 03:48:35.620856 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.619822    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-host-run-netns\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.620856 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.620133    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/503f1518-10bb-424b-87b0-00bf7cf2239f-hosts-file\") pod \"node-resolver-px55t\" (UID: \"503f1518-10bb-424b-87b0-00bf7cf2239f\") " pod="openshift-dns/node-resolver-px55t"
Sep 07 03:48:35.620856 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.620329    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-host-var-lib-kubelet\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.620856 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.620419    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52683f85-6093-473d-9e11-541e6ec8b2fb-host\") pod \"node-ca-xg5w5\" (UID: \"52683f85-6093-473d-9e11-541e6ec8b2fb\") " pod="openshift-image-registry/node-ca-xg5w5"
Sep 07 03:48:35.620856 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.620454    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/de17335a-7920-400b-ad6d-73fe656c84e0-cni-binary-copy\") pod \"multus-additional-cni-plugins-prd5g\" (UID: \"de17335a-7920-400b-ad6d-73fe656c84e0\") " pod="openshift-multus/multus-additional-cni-plugins-prd5g"
Sep 07 03:48:35.620856 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.620496    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-multus-conf-dir\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.620856 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.620582    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/de17335a-7920-400b-ad6d-73fe656c84e0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-prd5g\" (UID: \"de17335a-7920-400b-ad6d-73fe656c84e0\") " pod="openshift-multus/multus-additional-cni-plugins-prd5g"
Sep 07 03:48:35.620856 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.620774    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-etc-kubernetes\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.620856 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.620823    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-multus-cni-dir\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.621432 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.620970    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/de17335a-7920-400b-ad6d-73fe656c84e0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-prd5g\" (UID: \"de17335a-7920-400b-ad6d-73fe656c84e0\") " pod="openshift-multus/multus-additional-cni-plugins-prd5g"
Sep 07 03:48:35.621432 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.621103    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8c1667f2-8f99-44ed-96de-544fea7fe5a6-multus-daemon-config\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.621432 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.621131    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-host-var-lib-cni-multus\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.621432 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.621169    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/244e7a50-68b4-41dc-aa4b-8b83813cf489-dbus\") pod \"global-pull-secret-syncer-v9fzl\" (UID: \"244e7a50-68b4-41dc-aa4b-8b83813cf489\") " pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:35.621432 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.621216    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f5e84b44-6d3b-464b-a987-fff4a928f773-konnectivity-ca\") pod \"konnectivity-agent-gw9hg\" (UID: \"f5e84b44-6d3b-464b-a987-fff4a928f773\") " pod="kube-system/konnectivity-agent-gw9hg"
Sep 07 03:48:35.621432 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.621267    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fa0c971c-a802-448e-a954-eaa78c38cb79-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mbs8m\" (UID: \"fa0c971c-a802-448e-a954-eaa78c38cb79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m"
Sep 07 03:48:35.621432 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.621277    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de17335a-7920-400b-ad6d-73fe656c84e0-system-cni-dir\") pod \"multus-additional-cni-plugins-prd5g\" (UID: \"de17335a-7920-400b-ad6d-73fe656c84e0\") " pod="openshift-multus/multus-additional-cni-plugins-prd5g"
Sep 07 03:48:35.621432 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.621303    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fa0c971c-a802-448e-a954-eaa78c38cb79-etc-selinux\") pod \"aws-ebs-csi-driver-node-mbs8m\" (UID: \"fa0c971c-a802-448e-a954-eaa78c38cb79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m"
Sep 07 03:48:35.621432 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.621386    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-system-cni-dir\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.621892 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.621448    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fa0c971c-a802-448e-a954-eaa78c38cb79-sys-fs\") pod \"aws-ebs-csi-driver-node-mbs8m\" (UID: \"fa0c971c-a802-448e-a954-eaa78c38cb79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m"
Sep 07 03:48:35.621892 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.621516    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/244e7a50-68b4-41dc-aa4b-8b83813cf489-kubelet-config\") pod \"global-pull-secret-syncer-v9fzl\" (UID: \"244e7a50-68b4-41dc-aa4b-8b83813cf489\") " pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:35.621892 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.621532    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fa0c971c-a802-448e-a954-eaa78c38cb79-registration-dir\") pod \"aws-ebs-csi-driver-node-mbs8m\" (UID: \"fa0c971c-a802-448e-a954-eaa78c38cb79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m"
Sep 07 03:48:35.621892 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.621564    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8c1667f2-8f99-44ed-96de-544fea7fe5a6-cni-binary-copy\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.621892 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.621570    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-host-var-lib-cni-bin\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.621892 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:35.621646    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 03:48:35.621892 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:35.621702    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret podName:244e7a50-68b4-41dc-aa4b-8b83813cf489 nodeName:}" failed. No retries permitted until 2026-09-07 03:48:36.1216889 +0000 UTC m=+3.055169598 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret") pod "global-pull-secret-syncer-v9fzl" (UID: "244e7a50-68b4-41dc-aa4b-8b83813cf489") : object "kube-system"/"original-pull-secret" not registered
Sep 07 03:48:35.621892 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.621765    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/52683f85-6093-473d-9e11-541e6ec8b2fb-serviceca\") pod \"node-ca-xg5w5\" (UID: \"52683f85-6093-473d-9e11-541e6ec8b2fb\") " pod="openshift-image-registry/node-ca-xg5w5"
Sep 07 03:48:35.621892 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.621786    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fa0c971c-a802-448e-a954-eaa78c38cb79-socket-dir\") pod \"aws-ebs-csi-driver-node-mbs8m\" (UID: \"fa0c971c-a802-448e-a954-eaa78c38cb79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m"
Sep 07 03:48:35.622384 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.622011    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/de17335a-7920-400b-ad6d-73fe656c84e0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-prd5g\" (UID: \"de17335a-7920-400b-ad6d-73fe656c84e0\") " pod="openshift-multus/multus-additional-cni-plugins-prd5g"
Sep 07 03:48:35.622384 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.621570    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-hostroot\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.622384 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.622062    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/503f1518-10bb-424b-87b0-00bf7cf2239f-tmp-dir\") pod \"node-resolver-px55t\" (UID: \"503f1518-10bb-424b-87b0-00bf7cf2239f\") " pod="openshift-dns/node-resolver-px55t"
Sep 07 03:48:35.622384 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.622087    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-multus-socket-dir-parent\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.622384 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.622140    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/de17335a-7920-400b-ad6d-73fe656c84e0-cnibin\") pod \"multus-additional-cni-plugins-prd5g\" (UID: \"de17335a-7920-400b-ad6d-73fe656c84e0\") " pod="openshift-multus/multus-additional-cni-plugins-prd5g"
Sep 07 03:48:35.622384 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.622163    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-cnibin\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.622384 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.622226    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8c1667f2-8f99-44ed-96de-544fea7fe5a6-host-run-k8s-cni-cncf-io\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.622384 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.622361    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9ae1bb54-3861-4759-9041-c97a6bf9b6b5-iptables-alerter-script\") pod \"iptables-alerter-8mbdb\" (UID: \"9ae1bb54-3861-4759-9041-c97a6bf9b6b5\") " pod="openshift-network-operator/iptables-alerter-8mbdb"
Sep 07 03:48:35.626597 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:35.626574    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 03:48:35.626597 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:35.626600    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 03:48:35.626807 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:35.626613    2583 projected.go:196] Error preparing data for projected volume kube-api-access-kfcn7 for pod openshift-network-diagnostics/network-check-target-qbsbx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 03:48:35.626807 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:35.626669    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7 podName:4cf74d00-0e92-415e-8129-2033297d3707 nodeName:}" failed. No retries permitted until 2026-09-07 03:48:36.126654597 +0000 UTC m=+3.060135300 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kfcn7" (UniqueName: "kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7") pod "network-check-target-qbsbx" (UID: "4cf74d00-0e92-415e-8129-2033297d3707") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 03:48:35.626807 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.626779    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f5e84b44-6d3b-464b-a987-fff4a928f773-agent-certs\") pod \"konnectivity-agent-gw9hg\" (UID: \"f5e84b44-6d3b-464b-a987-fff4a928f773\") " pod="kube-system/konnectivity-agent-gw9hg"
Sep 07 03:48:35.629422 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.629395    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx8qm\" (UniqueName: \"kubernetes.io/projected/9ae1bb54-3861-4759-9041-c97a6bf9b6b5-kube-api-access-wx8qm\") pod \"iptables-alerter-8mbdb\" (UID: \"9ae1bb54-3861-4759-9041-c97a6bf9b6b5\") " pod="openshift-network-operator/iptables-alerter-8mbdb"
Sep 07 03:48:35.629422 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.629406    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsd4p\" (UniqueName: \"kubernetes.io/projected/503f1518-10bb-424b-87b0-00bf7cf2239f-kube-api-access-vsd4p\") pod \"node-resolver-px55t\" (UID: \"503f1518-10bb-424b-87b0-00bf7cf2239f\") " pod="openshift-dns/node-resolver-px55t"
Sep 07 03:48:35.630027 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.629997    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgwcp\" (UniqueName: \"kubernetes.io/projected/8c1667f2-8f99-44ed-96de-544fea7fe5a6-kube-api-access-cgwcp\") pod \"multus-9cztj\" (UID: \"8c1667f2-8f99-44ed-96de-544fea7fe5a6\") " pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.630619 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.630557    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n9x9\" (UniqueName: \"kubernetes.io/projected/de17335a-7920-400b-ad6d-73fe656c84e0-kube-api-access-4n9x9\") pod \"multus-additional-cni-plugins-prd5g\" (UID: \"de17335a-7920-400b-ad6d-73fe656c84e0\") " pod="openshift-multus/multus-additional-cni-plugins-prd5g"
Sep 07 03:48:35.631982 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.631958    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh8wl\" (UniqueName: \"kubernetes.io/projected/fa0c971c-a802-448e-a954-eaa78c38cb79-kube-api-access-sh8wl\") pod \"aws-ebs-csi-driver-node-mbs8m\" (UID: \"fa0c971c-a802-448e-a954-eaa78c38cb79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m"
Sep 07 03:48:35.632063 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.632022    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5jpz\" (UniqueName: \"kubernetes.io/projected/52683f85-6093-473d-9e11-541e6ec8b2fb-kube-api-access-d5jpz\") pod \"node-ca-xg5w5\" (UID: \"52683f85-6093-473d-9e11-541e6ec8b2fb\") " pod="openshift-image-registry/node-ca-xg5w5"
Sep 07 03:48:35.713452 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.713355    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jzp9s"
Sep 07 03:48:35.721450 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.721427    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:48:35.732520 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.732492    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gw9hg"
Sep 07 03:48:35.736655 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.736631    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-px55t"
Sep 07 03:48:35.743485 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.743453    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-prd5g"
Sep 07 03:48:35.751455 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.751426    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9cztj"
Sep 07 03:48:35.760395 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.760371    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8mbdb"
Sep 07 03:48:35.768446 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.768420    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xg5w5"
Sep 07 03:48:35.776441 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.776415    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m"
Sep 07 03:48:35.928580 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:35.928541    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 03:48:36.023669 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:36.023576    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs\") pod \"network-metrics-daemon-sml8b\" (UID: \"bb13bb24-ca38-4d13-aa0c-1c47eb028ae1\") " pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:48:36.023819 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:36.023729    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 03:48:36.023819 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:36.023794    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs podName:bb13bb24-ca38-4d13-aa0c-1c47eb028ae1 nodeName:}" failed. No retries permitted until 2026-09-07 03:48:37.02377744 +0000 UTC m=+3.957258138 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs") pod "network-metrics-daemon-sml8b" (UID: "bb13bb24-ca38-4d13-aa0c-1c47eb028ae1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 03:48:36.124235 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:36.124196    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret\") pod \"global-pull-secret-syncer-v9fzl\" (UID: \"244e7a50-68b4-41dc-aa4b-8b83813cf489\") " pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:36.124436 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:36.124389    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 03:48:36.124505 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:36.124484    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret podName:244e7a50-68b4-41dc-aa4b-8b83813cf489 nodeName:}" failed. No retries permitted until 2026-09-07 03:48:37.124458401 +0000 UTC m=+4.057939125 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret") pod "global-pull-secret-syncer-v9fzl" (UID: "244e7a50-68b4-41dc-aa4b-8b83813cf489") : object "kube-system"/"original-pull-secret" not registered
Sep 07 03:48:36.224807 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:36.224766    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfcn7\" (UniqueName: \"kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7\") pod \"network-check-target-qbsbx\" (UID: \"4cf74d00-0e92-415e-8129-2033297d3707\") " pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:48:36.224971 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:36.224947    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 03:48:36.225010 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:36.224979    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 03:48:36.225010 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:36.224996    2583 projected.go:196] Error preparing data for projected volume kube-api-access-kfcn7 for pod openshift-network-diagnostics/network-check-target-qbsbx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 03:48:36.225096 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:36.225067    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7 podName:4cf74d00-0e92-415e-8129-2033297d3707 nodeName:}" failed. No retries permitted until 2026-09-07 03:48:37.225046262 +0000 UTC m=+4.158526977 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kfcn7" (UniqueName: "kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7") pod "network-check-target-qbsbx" (UID: "4cf74d00-0e92-415e-8129-2033297d3707") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 03:48:36.452157 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:36.452076    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 03:43:36 +0000 UTC" deadline="2028-03-13 08:57:06.505586572 +0000 UTC"
Sep 07 03:48:36.452157 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:36.452153    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13277h8m30.053437325s"
Sep 07 03:48:36.494473 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:36.494429    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c1667f2_8f99_44ed_96de_544fea7fe5a6.slice/crio-97597aa8600b8fff987d8b9651509a5014a0de8ea5d808ca7f293c4075fef7f2 WatchSource:0}: Error finding container 97597aa8600b8fff987d8b9651509a5014a0de8ea5d808ca7f293c4075fef7f2: Status 404 returned error can't find the container with id 97597aa8600b8fff987d8b9651509a5014a0de8ea5d808ca7f293c4075fef7f2
Sep 07 03:48:36.495170 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:36.495143    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52683f85_6093_473d_9e11_541e6ec8b2fb.slice/crio-31120f800daab6f8a6d0f552e07ab05ecaee5dcdadeea1aab9bfaa690f359158 WatchSource:0}: Error finding container 31120f800daab6f8a6d0f552e07ab05ecaee5dcdadeea1aab9bfaa690f359158: Status 404 returned error can't find the container with id 31120f800daab6f8a6d0f552e07ab05ecaee5dcdadeea1aab9bfaa690f359158
Sep 07 03:48:36.496679 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:48:36.496649    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf1e9edf_b324_494e_aa39_b1351e2dd686.slice/crio-e6d24143fcaab3156c79c70b622537710ca9e3ab7e74267605d6c74de9747ba0 WatchSource:0}: Error finding container e6d24143fcaab3156c79c70b622537710ca9e3ab7e74267605d6c74de9747ba0: Status 404 returned error can't find the container with id e6d24143fcaab3156c79c70b622537710ca9e3ab7e74267605d6c74de9747ba0
Sep 07 03:48:36.515715 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:36.515694    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:36.515812 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:36.515790    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v9fzl" podUID="244e7a50-68b4-41dc-aa4b-8b83813cf489"
Sep 07 03:48:36.528549 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:36.528506    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh" event={"ID":"de89531d-86b1-404c-8140-2ffa057b333b","Type":"ContainerStarted","Data":"9742ecbf56e8613aa2cb8d7f374f21a27df2b5e3cb023c0d2cc6f3bd70808ec3"}
Sep 07 03:48:36.530180 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:36.530153    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m" event={"ID":"fa0c971c-a802-448e-a954-eaa78c38cb79","Type":"ContainerStarted","Data":"5626f593a67a126430e24be9544b88e029b530bed7a8fd842189e27c83838cd6"}
Sep 07 03:48:36.531571 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:36.531541    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prd5g" event={"ID":"de17335a-7920-400b-ad6d-73fe656c84e0","Type":"ContainerStarted","Data":"f7c6c0cddd1ed08410182f9b175f3c7b31f244b79a4bde3f73f94e993d244365"}
Sep 07 03:48:36.533035 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:36.533008    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xg5w5" event={"ID":"52683f85-6093-473d-9e11-541e6ec8b2fb","Type":"ContainerStarted","Data":"31120f800daab6f8a6d0f552e07ab05ecaee5dcdadeea1aab9bfaa690f359158"}
Sep 07 03:48:36.534333 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:36.534306    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8mbdb" event={"ID":"9ae1bb54-3861-4759-9041-c97a6bf9b6b5","Type":"ContainerStarted","Data":"36d1256547ee04870b193bced1f52257b495997e8207d81024163a1b039fd895"}
Sep 07 03:48:36.538883 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:36.538843    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-px55t" event={"ID":"503f1518-10bb-424b-87b0-00bf7cf2239f","Type":"ContainerStarted","Data":"8cea6e4b52b2b7ad6d34257960df40dcc7e92ede9ec2bc55d80d344818d424fe"}
Sep 07 03:48:36.540646 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:36.540620    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gw9hg" event={"ID":"f5e84b44-6d3b-464b-a987-fff4a928f773","Type":"ContainerStarted","Data":"42a8064b8f4e7f7c81ac56803350e9f2cca86e19def8afe8cc9b62666c2e6c93"}
Sep 07 03:48:36.545363 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:36.545341    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9cztj" event={"ID":"8c1667f2-8f99-44ed-96de-544fea7fe5a6","Type":"ContainerStarted","Data":"97597aa8600b8fff987d8b9651509a5014a0de8ea5d808ca7f293c4075fef7f2"}
Sep 07 03:48:36.546698 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:36.546674    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jzp9s" event={"ID":"af1e9edf-b324-494e-aa39-b1351e2dd686","Type":"ContainerStarted","Data":"e6d24143fcaab3156c79c70b622537710ca9e3ab7e74267605d6c74de9747ba0"}
Sep 07 03:48:37.032044 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:37.031949    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs\") pod \"network-metrics-daemon-sml8b\" (UID: \"bb13bb24-ca38-4d13-aa0c-1c47eb028ae1\") " pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:48:37.032212 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:37.032122    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 03:48:37.032212 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:37.032190    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs podName:bb13bb24-ca38-4d13-aa0c-1c47eb028ae1 nodeName:}" failed. No retries permitted until 2026-09-07 03:48:39.032169651 +0000 UTC m=+5.965650353 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs") pod "network-metrics-daemon-sml8b" (UID: "bb13bb24-ca38-4d13-aa0c-1c47eb028ae1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 03:48:37.133370 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:37.133323    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret\") pod \"global-pull-secret-syncer-v9fzl\" (UID: \"244e7a50-68b4-41dc-aa4b-8b83813cf489\") " pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:37.133572 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:37.133520    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 03:48:37.133643 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:37.133594    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret podName:244e7a50-68b4-41dc-aa4b-8b83813cf489 nodeName:}" failed. No retries permitted until 2026-09-07 03:48:39.133573716 +0000 UTC m=+6.067054428 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret") pod "global-pull-secret-syncer-v9fzl" (UID: "244e7a50-68b4-41dc-aa4b-8b83813cf489") : object "kube-system"/"original-pull-secret" not registered
Sep 07 03:48:37.234641 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:37.234569    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfcn7\" (UniqueName: \"kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7\") pod \"network-check-target-qbsbx\" (UID: \"4cf74d00-0e92-415e-8129-2033297d3707\") " pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:48:37.234921 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:37.234773    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 03:48:37.234921 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:37.234794    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 03:48:37.234921 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:37.234806    2583 projected.go:196] Error preparing data for projected volume kube-api-access-kfcn7 for pod openshift-network-diagnostics/network-check-target-qbsbx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 03:48:37.234921 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:37.234879    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7 podName:4cf74d00-0e92-415e-8129-2033297d3707 nodeName:}" failed. No retries permitted until 2026-09-07 03:48:39.234855839 +0000 UTC m=+6.168336538 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kfcn7" (UniqueName: "kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7") pod "network-check-target-qbsbx" (UID: "4cf74d00-0e92-415e-8129-2033297d3707") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 03:48:37.453223 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:37.453125    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 03:43:36 +0000 UTC" deadline="2028-04-11 15:33:15.529241842 +0000 UTC"
Sep 07 03:48:37.453223 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:37.453171    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13979h44m38.07607631s"
Sep 07 03:48:37.519149 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:37.518419    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:48:37.519149 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:37.518586    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sml8b" podUID="bb13bb24-ca38-4d13-aa0c-1c47eb028ae1"
Sep 07 03:48:37.519149 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:37.519002    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:48:37.519149 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:37.519095    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbsbx" podUID="4cf74d00-0e92-415e-8129-2033297d3707"
Sep 07 03:48:37.568986 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:37.568938    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-175.ec2.internal" event={"ID":"0a39a5eb3a918e1ff8bcbe08348a2d2c","Type":"ContainerStarted","Data":"7161fbcda8664ebeda333b9312f71471484cb1f24e2165dd4e17955d9d5d4b18"}
Sep 07 03:48:37.591071 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:37.590336    2583 generic.go:356] "Generic (PLEG): container finished" podID="d405fbd5d2fd527a362872a57cd14648" containerID="3f52d7ec1448dc8430fb1ddd126cc84e60e9956a2a2601440b27472b9cd1893a" exitCode=0
Sep 07 03:48:37.591071 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:37.590395    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-175.ec2.internal" event={"ID":"d405fbd5d2fd527a362872a57cd14648","Type":"ContainerDied","Data":"3f52d7ec1448dc8430fb1ddd126cc84e60e9956a2a2601440b27472b9cd1893a"}
Sep 07 03:48:37.606347 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:37.606272    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-175.ec2.internal" podStartSLOduration=2.606246918 podStartE2EDuration="2.606246918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 03:48:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:48:37.585525866 +0000 UTC m=+4.519006587" watchObservedRunningTime="2026-09-07 03:48:37.606246918 +0000 UTC m=+4.539727641"
Sep 07 03:48:38.515822 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:38.515784    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:38.516371 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:38.515944    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v9fzl" podUID="244e7a50-68b4-41dc-aa4b-8b83813cf489"
Sep 07 03:48:38.630887 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:38.630845    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-175.ec2.internal" event={"ID":"d405fbd5d2fd527a362872a57cd14648","Type":"ContainerStarted","Data":"fb70aeeb4bd902ec1ce692435b7744fa417d537f0559717f36b1e9b89e4c4d34"}
Sep 07 03:48:39.055745 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:39.055700    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs\") pod \"network-metrics-daemon-sml8b\" (UID: \"bb13bb24-ca38-4d13-aa0c-1c47eb028ae1\") " pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:48:39.055937 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:39.055891    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 03:48:39.056019 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:39.055962    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs podName:bb13bb24-ca38-4d13-aa0c-1c47eb028ae1 nodeName:}" failed. No retries permitted until 2026-09-07 03:48:43.055940254 +0000 UTC m=+9.989420969 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs") pod "network-metrics-daemon-sml8b" (UID: "bb13bb24-ca38-4d13-aa0c-1c47eb028ae1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 03:48:39.157018 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:39.156968    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret\") pod \"global-pull-secret-syncer-v9fzl\" (UID: \"244e7a50-68b4-41dc-aa4b-8b83813cf489\") " pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:39.157221 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:39.157163    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 03:48:39.157365 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:39.157230    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret podName:244e7a50-68b4-41dc-aa4b-8b83813cf489 nodeName:}" failed. No retries permitted until 2026-09-07 03:48:43.157211205 +0000 UTC m=+10.090691909 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret") pod "global-pull-secret-syncer-v9fzl" (UID: "244e7a50-68b4-41dc-aa4b-8b83813cf489") : object "kube-system"/"original-pull-secret" not registered
Sep 07 03:48:39.257933 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:39.257890    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfcn7\" (UniqueName: \"kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7\") pod \"network-check-target-qbsbx\" (UID: \"4cf74d00-0e92-415e-8129-2033297d3707\") " pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:48:39.258155 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:39.258119    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 03:48:39.258155 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:39.258148    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 03:48:39.258382 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:39.258162    2583 projected.go:196] Error preparing data for projected volume kube-api-access-kfcn7 for pod openshift-network-diagnostics/network-check-target-qbsbx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 03:48:39.258382 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:39.258228    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7 podName:4cf74d00-0e92-415e-8129-2033297d3707 nodeName:}" failed. No retries permitted until 2026-09-07 03:48:43.25820702 +0000 UTC m=+10.191687735 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kfcn7" (UniqueName: "kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7") pod "network-check-target-qbsbx" (UID: "4cf74d00-0e92-415e-8129-2033297d3707") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 03:48:39.519235 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:39.518353    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:48:39.519235 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:39.518496    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbsbx" podUID="4cf74d00-0e92-415e-8129-2033297d3707"
Sep 07 03:48:39.519235 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:39.518964    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:48:39.519235 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:39.519078    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sml8b" podUID="bb13bb24-ca38-4d13-aa0c-1c47eb028ae1"
Sep 07 03:48:40.516626 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:40.516066    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:40.516626 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:40.516212    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v9fzl" podUID="244e7a50-68b4-41dc-aa4b-8b83813cf489"
Sep 07 03:48:41.518120 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:41.517405    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:48:41.518120 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:41.517564    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sml8b" podUID="bb13bb24-ca38-4d13-aa0c-1c47eb028ae1"
Sep 07 03:48:41.518120 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:41.517960    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:48:41.518120 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:41.518065    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbsbx" podUID="4cf74d00-0e92-415e-8129-2033297d3707"
Sep 07 03:48:42.515811 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:42.515479    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:42.515998 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:42.515845    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v9fzl" podUID="244e7a50-68b4-41dc-aa4b-8b83813cf489"
Sep 07 03:48:43.090123 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:43.090076    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs\") pod \"network-metrics-daemon-sml8b\" (UID: \"bb13bb24-ca38-4d13-aa0c-1c47eb028ae1\") " pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:48:43.090654 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:43.090277    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 03:48:43.090654 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:43.090364    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs podName:bb13bb24-ca38-4d13-aa0c-1c47eb028ae1 nodeName:}" failed. No retries permitted until 2026-09-07 03:48:51.090340821 +0000 UTC m=+18.023821523 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs") pod "network-metrics-daemon-sml8b" (UID: "bb13bb24-ca38-4d13-aa0c-1c47eb028ae1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 03:48:43.191467 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:43.191419    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret\") pod \"global-pull-secret-syncer-v9fzl\" (UID: \"244e7a50-68b4-41dc-aa4b-8b83813cf489\") " pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:43.191671 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:43.191585    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 03:48:43.191671 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:43.191651    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret podName:244e7a50-68b4-41dc-aa4b-8b83813cf489 nodeName:}" failed. No retries permitted until 2026-09-07 03:48:51.191632087 +0000 UTC m=+18.125112784 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret") pod "global-pull-secret-syncer-v9fzl" (UID: "244e7a50-68b4-41dc-aa4b-8b83813cf489") : object "kube-system"/"original-pull-secret" not registered
Sep 07 03:48:43.292700 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:43.292591    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfcn7\" (UniqueName: \"kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7\") pod \"network-check-target-qbsbx\" (UID: \"4cf74d00-0e92-415e-8129-2033297d3707\") " pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:48:43.292897 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:43.292812    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 03:48:43.292897 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:43.292846    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 03:48:43.292897 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:43.292862    2583 projected.go:196] Error preparing data for projected volume kube-api-access-kfcn7 for pod openshift-network-diagnostics/network-check-target-qbsbx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 03:48:43.293068 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:43.292934    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7 podName:4cf74d00-0e92-415e-8129-2033297d3707 nodeName:}" failed. No retries permitted until 2026-09-07 03:48:51.292913784 +0000 UTC m=+18.226394502 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kfcn7" (UniqueName: "kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7") pod "network-check-target-qbsbx" (UID: "4cf74d00-0e92-415e-8129-2033297d3707") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 03:48:43.516725 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:43.516637    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:48:43.516893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:43.516722    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:48:43.516893 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:43.516752    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbsbx" podUID="4cf74d00-0e92-415e-8129-2033297d3707"
Sep 07 03:48:43.516893 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:43.516839    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sml8b" podUID="bb13bb24-ca38-4d13-aa0c-1c47eb028ae1"
Sep 07 03:48:44.515728 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:44.515695    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:44.516172 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:44.515847    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v9fzl" podUID="244e7a50-68b4-41dc-aa4b-8b83813cf489"
Sep 07 03:48:45.516946 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:45.516452    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:48:45.516946 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:45.516570    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sml8b" podUID="bb13bb24-ca38-4d13-aa0c-1c47eb028ae1"
Sep 07 03:48:45.517647 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:45.517572    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:48:45.517882 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:45.517830    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbsbx" podUID="4cf74d00-0e92-415e-8129-2033297d3707"
Sep 07 03:48:46.515623 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:46.515580    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:46.515811 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:46.515716    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v9fzl" podUID="244e7a50-68b4-41dc-aa4b-8b83813cf489"
Sep 07 03:48:47.515653 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:47.515608    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:48:47.516118 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:47.515741    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sml8b" podUID="bb13bb24-ca38-4d13-aa0c-1c47eb028ae1"
Sep 07 03:48:47.516118 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:47.515804    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:48:47.516118 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:47.515915    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbsbx" podUID="4cf74d00-0e92-415e-8129-2033297d3707"
Sep 07 03:48:48.515652 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:48.515613    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:48.516123 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:48.515734    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v9fzl" podUID="244e7a50-68b4-41dc-aa4b-8b83813cf489"
Sep 07 03:48:49.516051 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:49.516010    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:48:49.516051 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:49.516047    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:48:49.516534 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:49.516146    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbsbx" podUID="4cf74d00-0e92-415e-8129-2033297d3707"
Sep 07 03:48:49.516534 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:49.516298    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sml8b" podUID="bb13bb24-ca38-4d13-aa0c-1c47eb028ae1"
Sep 07 03:48:50.515535 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:50.515506    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:50.515774 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:50.515633    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v9fzl" podUID="244e7a50-68b4-41dc-aa4b-8b83813cf489"
Sep 07 03:48:51.156935 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:51.156897    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs\") pod \"network-metrics-daemon-sml8b\" (UID: \"bb13bb24-ca38-4d13-aa0c-1c47eb028ae1\") " pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:48:51.157339 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:51.157049    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 03:48:51.157339 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:51.157134    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs podName:bb13bb24-ca38-4d13-aa0c-1c47eb028ae1 nodeName:}" failed. No retries permitted until 2026-09-07 03:49:07.157110891 +0000 UTC m=+34.090591601 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs") pod "network-metrics-daemon-sml8b" (UID: "bb13bb24-ca38-4d13-aa0c-1c47eb028ae1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 03:48:51.257896 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:51.257854    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret\") pod \"global-pull-secret-syncer-v9fzl\" (UID: \"244e7a50-68b4-41dc-aa4b-8b83813cf489\") " pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:51.258061 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:51.258014    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 03:48:51.258118 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:51.258088    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret podName:244e7a50-68b4-41dc-aa4b-8b83813cf489 nodeName:}" failed. No retries permitted until 2026-09-07 03:49:07.25806957 +0000 UTC m=+34.191550281 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret") pod "global-pull-secret-syncer-v9fzl" (UID: "244e7a50-68b4-41dc-aa4b-8b83813cf489") : object "kube-system"/"original-pull-secret" not registered
Sep 07 03:48:51.359139 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:51.359108    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfcn7\" (UniqueName: \"kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7\") pod \"network-check-target-qbsbx\" (UID: \"4cf74d00-0e92-415e-8129-2033297d3707\") " pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:48:51.359297 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:51.359246    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 03:48:51.359297 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:51.359260    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 03:48:51.359297 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:51.359269    2583 projected.go:196] Error preparing data for projected volume kube-api-access-kfcn7 for pod openshift-network-diagnostics/network-check-target-qbsbx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 03:48:51.359429 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:51.359314    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7 podName:4cf74d00-0e92-415e-8129-2033297d3707 nodeName:}" failed. No retries permitted until 2026-09-07 03:49:07.359301349 +0000 UTC m=+34.292782044 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kfcn7" (UniqueName: "kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7") pod "network-check-target-qbsbx" (UID: "4cf74d00-0e92-415e-8129-2033297d3707") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 03:48:51.515440 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:51.515401    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:48:51.515614 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:51.515531    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sml8b" podUID="bb13bb24-ca38-4d13-aa0c-1c47eb028ae1"
Sep 07 03:48:51.515614 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:51.515588    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:48:51.515725 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:51.515701    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbsbx" podUID="4cf74d00-0e92-415e-8129-2033297d3707"
Sep 07 03:48:52.516081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:52.516040    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:52.516567 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:52.516185    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v9fzl" podUID="244e7a50-68b4-41dc-aa4b-8b83813cf489"
Sep 07 03:48:53.519194 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:53.519164    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:48:53.519622 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:53.519290    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbsbx" podUID="4cf74d00-0e92-415e-8129-2033297d3707"
Sep 07 03:48:53.519622 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:53.519365    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:48:53.519622 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:53.519463    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sml8b" podUID="bb13bb24-ca38-4d13-aa0c-1c47eb028ae1"
Sep 07 03:48:54.515946 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:54.515911    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:54.516120 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:54.516043    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v9fzl" podUID="244e7a50-68b4-41dc-aa4b-8b83813cf489"
Sep 07 03:48:55.515788 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:55.515620    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:48:55.516323 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:55.515858    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbsbx" podUID="4cf74d00-0e92-415e-8129-2033297d3707"
Sep 07 03:48:55.516323 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:55.516196    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:48:55.516323 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:55.516298    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sml8b" podUID="bb13bb24-ca38-4d13-aa0c-1c47eb028ae1"
Sep 07 03:48:55.671939 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:55.671914    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-px55t" event={"ID":"503f1518-10bb-424b-87b0-00bf7cf2239f","Type":"ContainerStarted","Data":"fa8218a0bd1573f2c16ade6d20af17b799b77ca80ac9b00535e981b5eebf0077"}
Sep 07 03:48:55.673333 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:55.673309    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gw9hg" event={"ID":"f5e84b44-6d3b-464b-a987-fff4a928f773","Type":"ContainerStarted","Data":"bd29db2e8c89fe4b3c4c8b14a2861c848cd3eb857e073a54497f585248431235"}
Sep 07 03:48:55.674903 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:55.674880    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9cztj" event={"ID":"8c1667f2-8f99-44ed-96de-544fea7fe5a6","Type":"ContainerStarted","Data":"b53e4ce1fce589bf2b023dfe3144294bf88edd17462d8668dac5bfc78d684d09"}
Sep 07 03:48:55.676453 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:55.676421    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jzp9s" event={"ID":"af1e9edf-b324-494e-aa39-b1351e2dd686","Type":"ContainerStarted","Data":"317b4d6396bad20d7fbd7e9cd68d4c8dbc2b9cb6aa7de723882b99ff280d7c60"}
Sep 07 03:48:55.678443 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:55.678424    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh" event={"ID":"de89531d-86b1-404c-8140-2ffa057b333b","Type":"ContainerStarted","Data":"2f5169a20542a4281d21704e9b27a9a9be317d03c1d0261f9669b7f76a9a1188"}
Sep 07 03:48:55.678549 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:55.678482    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh" event={"ID":"de89531d-86b1-404c-8140-2ffa057b333b","Type":"ContainerStarted","Data":"a8b20e441b772cc056db4e18ce84cb641fc44b40c4d65b5414b66f1857b86193"}
Sep 07 03:48:55.678549 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:55.678517    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh" event={"ID":"de89531d-86b1-404c-8140-2ffa057b333b","Type":"ContainerStarted","Data":"1f9aaf89a14a89cc084672b1df6baf2bee36602cf8c49fed22d5b72c08a7f1d4"}
Sep 07 03:48:55.679811 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:55.679793    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m" event={"ID":"fa0c971c-a802-448e-a954-eaa78c38cb79","Type":"ContainerStarted","Data":"ba422d359df9e7b504bb07ae0ff756318ea3e2b890dbfc4eac80471fc15ca05c"}
Sep 07 03:48:55.681382 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:55.681360    2583 generic.go:356] "Generic (PLEG): container finished" podID="de17335a-7920-400b-ad6d-73fe656c84e0" containerID="8d115aa1fff27d2733a4f79388b4709ae97d17d0a854359a06be66148abbc14b" exitCode=0
Sep 07 03:48:55.681624 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:55.681405    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prd5g" event={"ID":"de17335a-7920-400b-ad6d-73fe656c84e0","Type":"ContainerDied","Data":"8d115aa1fff27d2733a4f79388b4709ae97d17d0a854359a06be66148abbc14b"}
Sep 07 03:48:55.683112 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:55.683089    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xg5w5" event={"ID":"52683f85-6093-473d-9e11-541e6ec8b2fb","Type":"ContainerStarted","Data":"c594479ad68ffdd6c5acb7b50b0cf1325f264e7fefb29877b36dbfe3567045ac"}
Sep 07 03:48:55.688827 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:55.688774    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-px55t" podStartSLOduration=2.963478946 podStartE2EDuration="21.688760323s" totalImagesPullingTime="18.725281377s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 03:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:48:55.6886909 +0000 UTC m=+22.622171617" watchObservedRunningTime="2026-09-07 03:48:55.688760323 +0000 UTC m=+22.622241042"
Sep 07 03:48:55.689095 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:55.689077    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-175.ec2.internal" podStartSLOduration=20.689071791 podStartE2EDuration="20.689071791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 03:48:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:48:38.648209512 +0000 UTC m=+5.581690234" watchObservedRunningTime="2026-09-07 03:48:55.689071791 +0000 UTC m=+22.622552551"
Sep 07 03:48:55.703205 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:55.703159    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gw9hg" podStartSLOduration=2.978249915 podStartE2EDuration="21.703141027s" totalImagesPullingTime="18.724891112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:48:55.702578724 +0000 UTC m=+22.636059441" watchObservedRunningTime="2026-09-07 03:48:55.703141027 +0000 UTC m=+22.636621778"
Sep 07 03:48:55.719432 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:55.719365    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9cztj" podStartSLOduration=2.9050342369999997 podStartE2EDuration="21.719343051s" totalImagesPullingTime="18.814308814s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 03:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:48:55.718682842 +0000 UTC m=+22.652163559" watchObservedRunningTime="2026-09-07 03:48:55.719343051 +0000 UTC m=+22.652823770"
Sep 07 03:48:55.740585 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:55.740535    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jzp9s" podStartSLOduration=2.9899484750000003 podStartE2EDuration="21.740520257s" totalImagesPullingTime="18.750571782s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 03:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:48:55.740311456 +0000 UTC m=+22.673792174" watchObservedRunningTime="2026-09-07 03:48:55.740520257 +0000 UTC m=+22.674000975"
Sep 07 03:48:55.766481 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:55.766456    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:48:55.778454 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:55.778411    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xg5w5" podStartSLOduration=3.04728366 podStartE2EDuration="21.778395106s" totalImagesPullingTime="18.731111446s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 03:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:48:55.77799011 +0000 UTC m=+22.711470830" watchObservedRunningTime="2026-09-07 03:48:55.778395106 +0000 UTC m=+22.711875824"
Sep 07 03:48:55.947633 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:55.947605    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gw9hg"
Sep 07 03:48:55.947811 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:55.947721    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:48:55.948295 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:55.948272    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gw9hg"
Sep 07 03:48:56.003264 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:56.003161    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:48:56.163819 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:56.163781    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:48:56.515791 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:56.515720    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:56.515955 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:56.515835    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v9fzl" podUID="244e7a50-68b4-41dc-aa4b-8b83813cf489"
Sep 07 03:48:56.689139 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:56.689108    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh" event={"ID":"de89531d-86b1-404c-8140-2ffa057b333b","Type":"ContainerStarted","Data":"f08dbea703a6bbe5ef283db1a97049dd69a0695692b717fa750b3354ba7c4056"}
Sep 07 03:48:56.689283 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:56.689204    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh" event={"ID":"de89531d-86b1-404c-8140-2ffa057b333b","Type":"ContainerStarted","Data":"f7015c072e88cd27ebafbd6ae0ed1fdf7828665dfce82302e77f023257ca7469"}
Sep 07 03:48:56.689335 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:56.689282    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh" event={"ID":"de89531d-86b1-404c-8140-2ffa057b333b","Type":"ContainerStarted","Data":"9d18daa0bac98435601d3561c1c2b1bc425d2b1ea2006ec67da7914c4c523f28"}
Sep 07 03:48:56.690904 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:56.690860    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8mbdb" event={"ID":"9ae1bb54-3861-4759-9041-c97a6bf9b6b5","Type":"ContainerStarted","Data":"048a8dbdc599f7c924917f086e2e0309ae8b6fa6fc9cc5143df6ac277290acff"}
Sep 07 03:48:56.691712 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:56.691695    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gw9hg"
Sep 07 03:48:56.692310 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:56.692290    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gw9hg"
Sep 07 03:48:56.707991 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:56.707932    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8mbdb" podStartSLOduration=3.976892986 podStartE2EDuration="22.707910828s" totalImagesPullingTime="18.731017842s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 03:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:48:56.706742603 +0000 UTC m=+23.640223322" watchObservedRunningTime="2026-09-07 03:48:56.707910828 +0000 UTC m=+23.641391729"
Sep 07 03:48:56.736362 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:56.736332    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:48:56.788417 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:56.788180    2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 07 03:48:56.837227 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:56.837179    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:48:57.373060 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:57.373024    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:48:57.477737 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:57.477547    2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-07T03:48:56.788412857Z","UUID":"1c46fc4b-adaa-4aba-9a9f-780d13cda35c","Handler":null,"Name":"","Endpoint":""}
Sep 07 03:48:57.479258 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:57.479187    2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 07 03:48:57.479258 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:57.479253    2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 07 03:48:57.516440 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:57.516356    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:48:57.516956 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:57.516356    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:48:57.516956 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:57.516488    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sml8b" podUID="bb13bb24-ca38-4d13-aa0c-1c47eb028ae1"
Sep 07 03:48:57.516956 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:57.516577    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbsbx" podUID="4cf74d00-0e92-415e-8129-2033297d3707"
Sep 07 03:48:57.572897 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:57.572856    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:48:57.695489 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:57.695452    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m" event={"ID":"fa0c971c-a802-448e-a954-eaa78c38cb79","Type":"ContainerStarted","Data":"aafa82ee3bc2dc81b3bfe3bed3ebd8d3610d3055302f35a7c53db29994861403"}
Sep 07 03:48:57.964632 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:57.964541    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:48:58.172080 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:58.172034    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:48:58.273323 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:58.273199    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:48:58.516257 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:58.516201    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:48:58.516431 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:58.516325    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v9fzl" podUID="244e7a50-68b4-41dc-aa4b-8b83813cf489"
Sep 07 03:48:58.563766 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:58.563682    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:48:58.701906 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:58.701868    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh" event={"ID":"de89531d-86b1-404c-8140-2ffa057b333b","Type":"ContainerStarted","Data":"d5cbdc54528044aa7a4c17e13a36b0df76bd560883421e72851a87aa8d3932d0"}
Sep 07 03:48:58.704355 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:58.704324    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m" event={"ID":"fa0c971c-a802-448e-a954-eaa78c38cb79","Type":"ContainerStarted","Data":"e1efa70acd1ba46a86083795489a48f6a6eb8a0ec15c3b407d041f2748b9afb2"}
Sep 07 03:48:58.722533 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:58.722469    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mbs8m" podStartSLOduration=3.852318728 podStartE2EDuration="24.722446913s" totalImagesPullingTime="20.870128185s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 03:48:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:48:58.721536158 +0000 UTC m=+25.655016878" watchObservedRunningTime="2026-09-07 03:48:58.722446913 +0000 UTC m=+25.655927633"
Sep 07 03:48:59.165231 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:59.165160    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:48:59.516491 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:59.515913    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:48:59.516491 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:59.516076    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sml8b" podUID="bb13bb24-ca38-4d13-aa0c-1c47eb028ae1"
Sep 07 03:48:59.516491 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:59.516132    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:48:59.516491 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:48:59.516263    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbsbx" podUID="4cf74d00-0e92-415e-8129-2033297d3707"
Sep 07 03:48:59.567915 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:59.567857    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:48:59.764543 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:48:59.764498    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:00.363168 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:00.362989    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:00.515776 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:00.515745    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:49:00.515945 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:00.515841    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v9fzl" podUID="244e7a50-68b4-41dc-aa4b-8b83813cf489"
Sep 07 03:49:00.716376 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:00.716338    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh" event={"ID":"de89531d-86b1-404c-8140-2ffa057b333b","Type":"ContainerStarted","Data":"f94ec7756b3d0cb435921846ccc52fc0903e5fd59dd93c12a68e36c0b9b025e1"}
Sep 07 03:49:00.716841 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:00.716571    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:49:00.718523 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:00.718487    2583 generic.go:356] "Generic (PLEG): container finished" podID="de17335a-7920-400b-ad6d-73fe656c84e0" containerID="58a27260169b87f50ef34c1babddaa5e569fcadfc1780e3aa21ec1eb756a132d" exitCode=0
Sep 07 03:49:00.718653 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:00.718542    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prd5g" event={"ID":"de17335a-7920-400b-ad6d-73fe656c84e0","Type":"ContainerDied","Data":"58a27260169b87f50ef34c1babddaa5e569fcadfc1780e3aa21ec1eb756a132d"}
Sep 07 03:49:00.734137 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:00.734115    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:49:00.741624 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:00.741583    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh" podStartSLOduration=7.957056514 podStartE2EDuration="26.741567683s" totalImagesPullingTime="18.784511169s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 03:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:49:00.740510425 +0000 UTC m=+27.673991143" watchObservedRunningTime="2026-09-07 03:49:00.741567683 +0000 UTC m=+27.675048379"
Sep 07 03:49:00.964569 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:00.964539    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:01.172741 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:01.172676    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:01.367550 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:01.367505    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:01.516189 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:01.516146    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:49:01.516390 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:01.516288    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbsbx" podUID="4cf74d00-0e92-415e-8129-2033297d3707"
Sep 07 03:49:01.516390 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:01.516348    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:49:01.516506 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:01.516474    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sml8b" podUID="bb13bb24-ca38-4d13-aa0c-1c47eb028ae1"
Sep 07 03:49:01.722643 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:01.722613    2583 generic.go:356] "Generic (PLEG): container finished" podID="de17335a-7920-400b-ad6d-73fe656c84e0" containerID="0f444b360e0fdecb7f77466686b6d2e50dad9c0e4a3f4a5e390ac89ceaa4039a" exitCode=0
Sep 07 03:49:01.723241 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:01.722658    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prd5g" event={"ID":"de17335a-7920-400b-ad6d-73fe656c84e0","Type":"ContainerDied","Data":"0f444b360e0fdecb7f77466686b6d2e50dad9c0e4a3f4a5e390ac89ceaa4039a"}
Sep 07 03:49:01.723241 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:01.722774    2583 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 07 03:49:01.723369 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:01.723299    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:49:01.739044 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:01.739022    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:49:01.775938 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:01.775909    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:02.023067 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:02.022986    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:02.366489 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:02.366463    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:02.516276 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:02.516244    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:49:02.516440 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:02.516349    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v9fzl" podUID="244e7a50-68b4-41dc-aa4b-8b83813cf489"
Sep 07 03:49:02.580620 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:02.580535    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:02.727043 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:02.727010    2583 generic.go:356] "Generic (PLEG): container finished" podID="de17335a-7920-400b-ad6d-73fe656c84e0" containerID="30e23906a0297b145e4fa6220b022c0166be7c4708b4fabf01c46183f588b94b" exitCode=0
Sep 07 03:49:02.727569 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:02.727088    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prd5g" event={"ID":"de17335a-7920-400b-ad6d-73fe656c84e0","Type":"ContainerDied","Data":"30e23906a0297b145e4fa6220b022c0166be7c4708b4fabf01c46183f588b94b"}
Sep 07 03:49:02.727569 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:02.727144    2583 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 07 03:49:02.769447 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:02.769417    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:02.982669 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:02.982638    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:49:03.519125 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:03.519093    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:49:03.519316 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:03.519202    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbsbx" podUID="4cf74d00-0e92-415e-8129-2033297d3707"
Sep 07 03:49:03.519316 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:03.519267    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:49:03.519411 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:03.519388    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sml8b" podUID="bb13bb24-ca38-4d13-aa0c-1c47eb028ae1"
Sep 07 03:49:04.515700 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:04.515671    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:49:04.516097 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:04.515823    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v9fzl" podUID="244e7a50-68b4-41dc-aa4b-8b83813cf489"
Sep 07 03:49:05.515573 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:05.515537    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:49:05.515773 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:05.515671    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sml8b" podUID="bb13bb24-ca38-4d13-aa0c-1c47eb028ae1"
Sep 07 03:49:05.515773 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:05.515736    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:49:05.516330 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:05.515839    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbsbx" podUID="4cf74d00-0e92-415e-8129-2033297d3707"
Sep 07 03:49:06.515902 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:06.515867    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:49:06.516397 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:06.516004    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v9fzl" podUID="244e7a50-68b4-41dc-aa4b-8b83813cf489"
Sep 07 03:49:06.605458 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:06.605417    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:06.657259 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:06.657225    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:07.173383 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:07.173343    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs\") pod \"network-metrics-daemon-sml8b\" (UID: \"bb13bb24-ca38-4d13-aa0c-1c47eb028ae1\") " pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:49:07.173548 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:07.173484    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 03:49:07.173589 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:07.173547    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs podName:bb13bb24-ca38-4d13-aa0c-1c47eb028ae1 nodeName:}" failed. No retries permitted until 2026-09-07 03:49:39.173530494 +0000 UTC m=+66.107011190 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs") pod "network-metrics-daemon-sml8b" (UID: "bb13bb24-ca38-4d13-aa0c-1c47eb028ae1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 03:49:07.274453 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:07.274424    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret\") pod \"global-pull-secret-syncer-v9fzl\" (UID: \"244e7a50-68b4-41dc-aa4b-8b83813cf489\") " pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:49:07.274627 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:07.274605    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 03:49:07.274706 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:07.274694    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret podName:244e7a50-68b4-41dc-aa4b-8b83813cf489 nodeName:}" failed. No retries permitted until 2026-09-07 03:49:39.274672962 +0000 UTC m=+66.208153677 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret") pod "global-pull-secret-syncer-v9fzl" (UID: "244e7a50-68b4-41dc-aa4b-8b83813cf489") : object "kube-system"/"original-pull-secret" not registered
Sep 07 03:49:07.375256 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:07.375189    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfcn7\" (UniqueName: \"kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7\") pod \"network-check-target-qbsbx\" (UID: \"4cf74d00-0e92-415e-8129-2033297d3707\") " pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:49:07.375423 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:07.375363    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 03:49:07.375423 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:07.375389    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 03:49:07.375423 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:07.375399    2583 projected.go:196] Error preparing data for projected volume kube-api-access-kfcn7 for pod openshift-network-diagnostics/network-check-target-qbsbx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 03:49:07.375549 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:07.375453    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7 podName:4cf74d00-0e92-415e-8129-2033297d3707 nodeName:}" failed. No retries permitted until 2026-09-07 03:49:39.375439261 +0000 UTC m=+66.308919957 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kfcn7" (UniqueName: "kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7") pod "network-check-target-qbsbx" (UID: "4cf74d00-0e92-415e-8129-2033297d3707") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 03:49:07.519127 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:07.519096    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:49:07.519529 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:07.519194    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:49:07.519529 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:07.519202    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sml8b" podUID="bb13bb24-ca38-4d13-aa0c-1c47eb028ae1"
Sep 07 03:49:07.519529 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:07.519288    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbsbx" podUID="4cf74d00-0e92-415e-8129-2033297d3707"
Sep 07 03:49:07.519529 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:07.519308    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:49:07.519529 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:07.519377    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v9fzl" podUID="244e7a50-68b4-41dc-aa4b-8b83813cf489"
Sep 07 03:49:07.741194 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:07.741112    2583 generic.go:356] "Generic (PLEG): container finished" podID="de17335a-7920-400b-ad6d-73fe656c84e0" containerID="a11f2748d7662380c0672de57476cb64b19680b771da207cb651ec1c1fc8d091" exitCode=0
Sep 07 03:49:07.741194 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:07.741164    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prd5g" event={"ID":"de17335a-7920-400b-ad6d-73fe656c84e0","Type":"ContainerDied","Data":"a11f2748d7662380c0672de57476cb64b19680b771da207cb651ec1c1fc8d091"}
Sep 07 03:49:08.747076 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:08.747041    2583 generic.go:356] "Generic (PLEG): container finished" podID="de17335a-7920-400b-ad6d-73fe656c84e0" containerID="cda9b994cf264616e3dbf8fbecafdf09f8290ce69aa82a9891bcc5f35d40ecec" exitCode=0
Sep 07 03:49:08.747593 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:08.747103    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prd5g" event={"ID":"de17335a-7920-400b-ad6d-73fe656c84e0","Type":"ContainerDied","Data":"cda9b994cf264616e3dbf8fbecafdf09f8290ce69aa82a9891bcc5f35d40ecec"}
Sep 07 03:49:09.518441 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:09.518414    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:49:09.518622 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:09.518556    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:49:09.518622 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:09.518581    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:49:09.518622 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:09.518573    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v9fzl" podUID="244e7a50-68b4-41dc-aa4b-8b83813cf489"
Sep 07 03:49:09.518797 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:09.518650    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbsbx" podUID="4cf74d00-0e92-415e-8129-2033297d3707"
Sep 07 03:49:09.518797 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:09.518725    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sml8b" podUID="bb13bb24-ca38-4d13-aa0c-1c47eb028ae1"
Sep 07 03:49:09.618016 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:09.617988    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:09.707394 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:09.707363    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:09.752836 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:09.752803    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-prd5g" event={"ID":"de17335a-7920-400b-ad6d-73fe656c84e0","Type":"ContainerStarted","Data":"28103bccb9a568fa6b2838c55f2fc662008dceea80a38410c634f7ae214d1b5e"}
Sep 07 03:49:09.785938 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:09.785840    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-prd5g" podStartSLOduration=6.875998629 podStartE2EDuration="35.785821403s" totalImagesPullingTime="28.909822774s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 03:48:34 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:49:09.785641554 +0000 UTC m=+36.719122274" watchObservedRunningTime="2026-09-07 03:49:09.785821403 +0000 UTC m=+36.719302121"
Sep 07 03:49:09.811351 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:09.811328    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:09.910504 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:09.910469    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:10.014228 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:10.014179    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:10.573912 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:10.573880    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:10.630473 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:10.630436    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:10.954373 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:10.954161    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:11.054506 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:11.054478    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:11.236395 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:11.236318    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qbsbx"]
Sep 07 03:49:11.236547 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:11.236427    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:49:11.236547 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:11.236523    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbsbx" podUID="4cf74d00-0e92-415e-8129-2033297d3707"
Sep 07 03:49:11.239578 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:11.239252    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sml8b"]
Sep 07 03:49:11.239578 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:11.239358    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:49:11.239578 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:11.239448    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sml8b" podUID="bb13bb24-ca38-4d13-aa0c-1c47eb028ae1"
Sep 07 03:49:11.251122 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:11.251096    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-v9fzl"]
Sep 07 03:49:11.251257 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:11.251201    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:49:11.251359 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:11.251330    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v9fzl" podUID="244e7a50-68b4-41dc-aa4b-8b83813cf489"
Sep 07 03:49:11.766683 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:11.766648    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:12.177458 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:12.177346    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:12.277864 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:12.277829    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:12.515372 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:12.515340    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:49:12.515543 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:12.515445    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sml8b" podUID="bb13bb24-ca38-4d13-aa0c-1c47eb028ae1"
Sep 07 03:49:13.518650 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:13.518624    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:49:13.518650 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:13.518642    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:49:13.519041 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:13.518710    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbsbx" podUID="4cf74d00-0e92-415e-8129-2033297d3707"
Sep 07 03:49:13.519041 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:13.518875    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v9fzl" podUID="244e7a50-68b4-41dc-aa4b-8b83813cf489"
Sep 07 03:49:14.515284 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:14.515247    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:49:14.515453 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:14.515359    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sml8b" podUID="bb13bb24-ca38-4d13-aa0c-1c47eb028ae1"
Sep 07 03:49:15.519085 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.519055    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:49:15.519454 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.519063    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:49:15.519454 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:15.519149    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v9fzl" podUID="244e7a50-68b4-41dc-aa4b-8b83813cf489"
Sep 07 03:49:15.519454 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:15.519235    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qbsbx" podUID="4cf74d00-0e92-415e-8129-2033297d3707"
Sep 07 03:49:15.868310 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.868282    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-175.ec2.internal" event="NodeReady"
Sep 07 03:49:15.868469 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.868409    2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 07 03:49:15.911118 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.911085    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gcl7p"]
Sep 07 03:49:15.916513 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.916486    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-k7hdq"]
Sep 07 03:49:15.916651 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.916608    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gcl7p"
Sep 07 03:49:15.918939 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.918914    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 07 03:49:15.919048 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.918923    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 07 03:49:15.919184 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.919171    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-n9q8q\""
Sep 07 03:49:15.926543 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.926521    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gcl7p"]
Sep 07 03:49:15.926640 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.926626    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-k7hdq"
Sep 07 03:49:15.929122 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.929106    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-b5xmc\""
Sep 07 03:49:15.929430 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.929413    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 07 03:49:15.929531 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.929428    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 07 03:49:15.929531 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.929468    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 07 03:49:15.932985 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.932965    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nmxn8"]
Sep 07 03:49:15.938755 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.938736    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nmxn8"
Sep 07 03:49:15.940569 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.940546    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-k7hdq"]
Sep 07 03:49:15.941448 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.941413    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 07 03:49:15.941448 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.941440    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 07 03:49:15.941625 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.941470    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7k4qm\""
Sep 07 03:49:15.941625 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.941419    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 07 03:49:15.941625 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.941559    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 07 03:49:15.947610 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:15.947588    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nmxn8"]
Sep 07 03:49:16.030648 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.030545    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:16.035466 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.035449    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f35afb94-a0dc-4039-8d88-81f6fc8326c0-config-volume\") pod \"dns-default-gcl7p\" (UID: \"f35afb94-a0dc-4039-8d88-81f6fc8326c0\") " pod="openshift-dns/dns-default-gcl7p"
Sep 07 03:49:16.035527 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.035476    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1a092764-0c63-4c0b-8456-a05adb098222-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nmxn8\" (UID: \"1a092764-0c63-4c0b-8456-a05adb098222\") " pod="openshift-insights/insights-runtime-extractor-nmxn8"
Sep 07 03:49:16.035527 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.035495    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sffxb\" (UniqueName: \"kubernetes.io/projected/1a092764-0c63-4c0b-8456-a05adb098222-kube-api-access-sffxb\") pod \"insights-runtime-extractor-nmxn8\" (UID: \"1a092764-0c63-4c0b-8456-a05adb098222\") " pod="openshift-insights/insights-runtime-extractor-nmxn8"
Sep 07 03:49:16.035527 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.035510    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f35afb94-a0dc-4039-8d88-81f6fc8326c0-metrics-tls\") pod \"dns-default-gcl7p\" (UID: \"f35afb94-a0dc-4039-8d88-81f6fc8326c0\") " pod="openshift-dns/dns-default-gcl7p"
Sep 07 03:49:16.035657 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.035540    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44zzb\" (UniqueName: \"kubernetes.io/projected/f35afb94-a0dc-4039-8d88-81f6fc8326c0-kube-api-access-44zzb\") pod \"dns-default-gcl7p\" (UID: \"f35afb94-a0dc-4039-8d88-81f6fc8326c0\") " pod="openshift-dns/dns-default-gcl7p"
Sep 07 03:49:16.035657 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.035570    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1a092764-0c63-4c0b-8456-a05adb098222-crio-socket\") pod \"insights-runtime-extractor-nmxn8\" (UID: \"1a092764-0c63-4c0b-8456-a05adb098222\") " pod="openshift-insights/insights-runtime-extractor-nmxn8"
Sep 07 03:49:16.035657 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.035585    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f35afb94-a0dc-4039-8d88-81f6fc8326c0-tmp-dir\") pod \"dns-default-gcl7p\" (UID: \"f35afb94-a0dc-4039-8d88-81f6fc8326c0\") " pod="openshift-dns/dns-default-gcl7p"
Sep 07 03:49:16.035657 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.035601    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a092764-0c63-4c0b-8456-a05adb098222-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nmxn8\" (UID: \"1a092764-0c63-4c0b-8456-a05adb098222\") " pod="openshift-insights/insights-runtime-extractor-nmxn8"
Sep 07 03:49:16.035657 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.035616    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/822aab36-17d2-448a-a74c-273fe4450e29-cert\") pod \"ingress-canary-k7hdq\" (UID: \"822aab36-17d2-448a-a74c-273fe4450e29\") " pod="openshift-ingress-canary/ingress-canary-k7hdq"
Sep 07 03:49:16.035657 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.035641    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgvn2\" (UniqueName: \"kubernetes.io/projected/822aab36-17d2-448a-a74c-273fe4450e29-kube-api-access-tgvn2\") pod \"ingress-canary-k7hdq\" (UID: \"822aab36-17d2-448a-a74c-273fe4450e29\") " pod="openshift-ingress-canary/ingress-canary-k7hdq"
Sep 07 03:49:16.035657 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.035657    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a092764-0c63-4c0b-8456-a05adb098222-data-volume\") pod \"insights-runtime-extractor-nmxn8\" (UID: \"1a092764-0c63-4c0b-8456-a05adb098222\") " pod="openshift-insights/insights-runtime-extractor-nmxn8"
Sep 07 03:49:16.087701 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.087650    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:16.136285 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.136250    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgvn2\" (UniqueName: \"kubernetes.io/projected/822aab36-17d2-448a-a74c-273fe4450e29-kube-api-access-tgvn2\") pod \"ingress-canary-k7hdq\" (UID: \"822aab36-17d2-448a-a74c-273fe4450e29\") " pod="openshift-ingress-canary/ingress-canary-k7hdq"
Sep 07 03:49:16.136404 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.136294    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a092764-0c63-4c0b-8456-a05adb098222-data-volume\") pod \"insights-runtime-extractor-nmxn8\" (UID: \"1a092764-0c63-4c0b-8456-a05adb098222\") " pod="openshift-insights/insights-runtime-extractor-nmxn8"
Sep 07 03:49:16.136404 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.136362    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f35afb94-a0dc-4039-8d88-81f6fc8326c0-config-volume\") pod \"dns-default-gcl7p\" (UID: \"f35afb94-a0dc-4039-8d88-81f6fc8326c0\") " pod="openshift-dns/dns-default-gcl7p"
Sep 07 03:49:16.136404 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.136388    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1a092764-0c63-4c0b-8456-a05adb098222-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nmxn8\" (UID: \"1a092764-0c63-4c0b-8456-a05adb098222\") " pod="openshift-insights/insights-runtime-extractor-nmxn8"
Sep 07 03:49:16.136540 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.136411    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sffxb\" (UniqueName: \"kubernetes.io/projected/1a092764-0c63-4c0b-8456-a05adb098222-kube-api-access-sffxb\") pod \"insights-runtime-extractor-nmxn8\" (UID: \"1a092764-0c63-4c0b-8456-a05adb098222\") " pod="openshift-insights/insights-runtime-extractor-nmxn8"
Sep 07 03:49:16.136540 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.136434    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f35afb94-a0dc-4039-8d88-81f6fc8326c0-metrics-tls\") pod \"dns-default-gcl7p\" (UID: \"f35afb94-a0dc-4039-8d88-81f6fc8326c0\") " pod="openshift-dns/dns-default-gcl7p"
Sep 07 03:49:16.136540 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.136458    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44zzb\" (UniqueName: \"kubernetes.io/projected/f35afb94-a0dc-4039-8d88-81f6fc8326c0-kube-api-access-44zzb\") pod \"dns-default-gcl7p\" (UID: \"f35afb94-a0dc-4039-8d88-81f6fc8326c0\") " pod="openshift-dns/dns-default-gcl7p"
Sep 07 03:49:16.136540 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.136495    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1a092764-0c63-4c0b-8456-a05adb098222-crio-socket\") pod \"insights-runtime-extractor-nmxn8\" (UID: \"1a092764-0c63-4c0b-8456-a05adb098222\") " pod="openshift-insights/insights-runtime-extractor-nmxn8"
Sep 07 03:49:16.136540 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.136519    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f35afb94-a0dc-4039-8d88-81f6fc8326c0-tmp-dir\") pod \"dns-default-gcl7p\" (UID: \"f35afb94-a0dc-4039-8d88-81f6fc8326c0\") " pod="openshift-dns/dns-default-gcl7p"
Sep 07 03:49:16.136744 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.136544    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a092764-0c63-4c0b-8456-a05adb098222-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nmxn8\" (UID: \"1a092764-0c63-4c0b-8456-a05adb098222\") " pod="openshift-insights/insights-runtime-extractor-nmxn8"
Sep 07 03:49:16.136744 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.136566    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/822aab36-17d2-448a-a74c-273fe4450e29-cert\") pod \"ingress-canary-k7hdq\" (UID: \"822aab36-17d2-448a-a74c-273fe4450e29\") " pod="openshift-ingress-canary/ingress-canary-k7hdq"
Sep 07 03:49:16.136904 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.136878    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1a092764-0c63-4c0b-8456-a05adb098222-crio-socket\") pod \"insights-runtime-extractor-nmxn8\" (UID: \"1a092764-0c63-4c0b-8456-a05adb098222\") " pod="openshift-insights/insights-runtime-extractor-nmxn8"
Sep 07 03:49:16.137082 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.137062    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f35afb94-a0dc-4039-8d88-81f6fc8326c0-tmp-dir\") pod \"dns-default-gcl7p\" (UID: \"f35afb94-a0dc-4039-8d88-81f6fc8326c0\") " pod="openshift-dns/dns-default-gcl7p"
Sep 07 03:49:16.137311 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.137291    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f35afb94-a0dc-4039-8d88-81f6fc8326c0-config-volume\") pod \"dns-default-gcl7p\" (UID: \"f35afb94-a0dc-4039-8d88-81f6fc8326c0\") " pod="openshift-dns/dns-default-gcl7p"
Sep 07 03:49:16.137394 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.137314    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1a092764-0c63-4c0b-8456-a05adb098222-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nmxn8\" (UID: \"1a092764-0c63-4c0b-8456-a05adb098222\") " pod="openshift-insights/insights-runtime-extractor-nmxn8"
Sep 07 03:49:16.137394 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.137381    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a092764-0c63-4c0b-8456-a05adb098222-data-volume\") pod \"insights-runtime-extractor-nmxn8\" (UID: \"1a092764-0c63-4c0b-8456-a05adb098222\") " pod="openshift-insights/insights-runtime-extractor-nmxn8"
Sep 07 03:49:16.140587 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.140567    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f35afb94-a0dc-4039-8d88-81f6fc8326c0-metrics-tls\") pod \"dns-default-gcl7p\" (UID: \"f35afb94-a0dc-4039-8d88-81f6fc8326c0\") " pod="openshift-dns/dns-default-gcl7p"
Sep 07 03:49:16.140690 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.140635    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a092764-0c63-4c0b-8456-a05adb098222-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nmxn8\" (UID: \"1a092764-0c63-4c0b-8456-a05adb098222\") " pod="openshift-insights/insights-runtime-extractor-nmxn8"
Sep 07 03:49:16.140766 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.140750    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/822aab36-17d2-448a-a74c-273fe4450e29-cert\") pod \"ingress-canary-k7hdq\" (UID: \"822aab36-17d2-448a-a74c-273fe4450e29\") " pod="openshift-ingress-canary/ingress-canary-k7hdq"
Sep 07 03:49:16.144306 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.144282    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sffxb\" (UniqueName: \"kubernetes.io/projected/1a092764-0c63-4c0b-8456-a05adb098222-kube-api-access-sffxb\") pod \"insights-runtime-extractor-nmxn8\" (UID: \"1a092764-0c63-4c0b-8456-a05adb098222\") " pod="openshift-insights/insights-runtime-extractor-nmxn8"
Sep 07 03:49:16.144410 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.144334    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44zzb\" (UniqueName: \"kubernetes.io/projected/f35afb94-a0dc-4039-8d88-81f6fc8326c0-kube-api-access-44zzb\") pod \"dns-default-gcl7p\" (UID: \"f35afb94-a0dc-4039-8d88-81f6fc8326c0\") " pod="openshift-dns/dns-default-gcl7p"
Sep 07 03:49:16.144802 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.144781    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgvn2\" (UniqueName: \"kubernetes.io/projected/822aab36-17d2-448a-a74c-273fe4450e29-kube-api-access-tgvn2\") pod \"ingress-canary-k7hdq\" (UID: \"822aab36-17d2-448a-a74c-273fe4450e29\") " pod="openshift-ingress-canary/ingress-canary-k7hdq"
Sep 07 03:49:16.227157 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.227116    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gcl7p"
Sep 07 03:49:16.235281 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.235256    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-k7hdq"
Sep 07 03:49:16.247558 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.247535    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nmxn8"
Sep 07 03:49:16.516169 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.516133    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:49:16.519351 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.519329    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-25ldn\""
Sep 07 03:49:16.519905 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.519328    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 07 03:49:16.566188 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.566116    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-k7hdq"]
Sep 07 03:49:16.569606 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.569581    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gcl7p"]
Sep 07 03:49:16.570132 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:49:16.570105    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod822aab36_17d2_448a_a74c_273fe4450e29.slice/crio-36e5601b08999c9135c9492e5a4c0274e43ce0cf2521702b79edfa1ccfe595e2 WatchSource:0}: Error finding container 36e5601b08999c9135c9492e5a4c0274e43ce0cf2521702b79edfa1ccfe595e2: Status 404 returned error can't find the container with id 36e5601b08999c9135c9492e5a4c0274e43ce0cf2521702b79edfa1ccfe595e2
Sep 07 03:49:16.588160 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.588130    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nmxn8"]
Sep 07 03:49:16.592687 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:49:16.592655    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a092764_0c63_4c0b_8456_a05adb098222.slice/crio-cb96e55d83ef022348ae1fa4c78c2eea4763ab4723699e578fd06f2c0e0a7fb6 WatchSource:0}: Error finding container cb96e55d83ef022348ae1fa4c78c2eea4763ab4723699e578fd06f2c0e0a7fb6: Status 404 returned error can't find the container with id cb96e55d83ef022348ae1fa4c78c2eea4763ab4723699e578fd06f2c0e0a7fb6
Sep 07 03:49:16.772190 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.772152    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nmxn8" event={"ID":"1a092764-0c63-4c0b-8456-a05adb098222","Type":"ContainerStarted","Data":"7fd4713c7e19e94882288e55b287dbec612fe442e87c9087348daccc7c26839b"}
Sep 07 03:49:16.772380 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.772236    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nmxn8" event={"ID":"1a092764-0c63-4c0b-8456-a05adb098222","Type":"ContainerStarted","Data":"cb96e55d83ef022348ae1fa4c78c2eea4763ab4723699e578fd06f2c0e0a7fb6"}
Sep 07 03:49:16.773309 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.773284    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-k7hdq" event={"ID":"822aab36-17d2-448a-a74c-273fe4450e29","Type":"ContainerStarted","Data":"36e5601b08999c9135c9492e5a4c0274e43ce0cf2521702b79edfa1ccfe595e2"}
Sep 07 03:49:16.774393 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:16.774370    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gcl7p" event={"ID":"f35afb94-a0dc-4039-8d88-81f6fc8326c0","Type":"ContainerStarted","Data":"dfdbe07bc8ad558e3cf97c5de849795243f5601cee016c2a53f0c1da7cb5108c"}
Sep 07 03:49:17.520588 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:17.520295    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:49:17.521354 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:17.520317    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:49:17.523602 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:17.523576    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 07 03:49:17.523602 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:17.523592    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 07 03:49:17.524570 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:17.524550    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 07 03:49:17.524689 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:17.524550    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wrs9m\""
Sep 07 03:49:17.615617 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:17.615580    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:17.715308 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:17.715255    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:19.785711 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:19.785672    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gcl7p" event={"ID":"f35afb94-a0dc-4039-8d88-81f6fc8326c0","Type":"ContainerStarted","Data":"4e3cc4dffa0fd90fe77842bb2d21d7a5b7ce26fd0e67a79e613d9a9822f6ec71"}
Sep 07 03:49:19.787721 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:19.787692    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nmxn8" event={"ID":"1a092764-0c63-4c0b-8456-a05adb098222","Type":"ContainerStarted","Data":"492975257be1eb1b4b3bf7025370b4a4f27e6ac464855bac4ae6cf105b03bed1"}
Sep 07 03:49:19.789195 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:19.789173    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-k7hdq" event={"ID":"822aab36-17d2-448a-a74c-273fe4450e29","Type":"ContainerStarted","Data":"3807af649b6e005ffdff04e1471588be9344644fefa3181d2aae8c0fe51dad41"}
Sep 07 03:49:19.805249 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:19.805076    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-k7hdq" podStartSLOduration=1.8797149499999999 podStartE2EDuration="4.805052277s" totalImagesPullingTime="2.925337327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:49:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:49:19.804871767 +0000 UTC m=+46.738352477" watchObservedRunningTime="2026-09-07 03:49:19.805052277 +0000 UTC m=+46.738533073"
Sep 07 03:49:19.982268 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:19.982235    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:20.083564 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.083470    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:20.151441 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.151407    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rdgxm"]
Sep 07 03:49:20.151787 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.151751    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:20.169017 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.168989    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rdgxm"]
Sep 07 03:49:20.169155 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.169095    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rdgxm"
Sep 07 03:49:20.171500 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.171476    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 07 03:49:20.171612 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.171507    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-42lcf\""
Sep 07 03:49:20.261351 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.261317    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:20.269487 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.269453    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fe79a6a4-0529-4ff4-aabd-7c0a152f2878-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-rdgxm\" (UID: \"fe79a6a4-0529-4ff4-aabd-7c0a152f2878\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rdgxm"
Sep 07 03:49:20.370575 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.370493    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fe79a6a4-0529-4ff4-aabd-7c0a152f2878-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-rdgxm\" (UID: \"fe79a6a4-0529-4ff4-aabd-7c0a152f2878\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rdgxm"
Sep 07 03:49:20.374849 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.374823    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fe79a6a4-0529-4ff4-aabd-7c0a152f2878-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-rdgxm\" (UID: \"fe79a6a4-0529-4ff4-aabd-7c0a152f2878\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rdgxm"
Sep 07 03:49:20.479304 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.479261    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rdgxm"
Sep 07 03:49:20.548985 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.548395    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-lkmpv"]
Sep 07 03:49:20.563135 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.563105    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-lkmpv"]
Sep 07 03:49:20.563401 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.563246    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-lkmpv"
Sep 07 03:49:20.566762 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.566737    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-vv579\""
Sep 07 03:49:20.566910 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.566855    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 07 03:49:20.566974 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.566945    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 07 03:49:20.568001 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.567903    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:20.664522 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.664265    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:20.672884 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.672845    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m84dp\" (UniqueName: \"kubernetes.io/projected/8d993695-6e46-4b29-98da-6ba5256072a3-kube-api-access-m84dp\") pod \"downloads-85d98c998f-lkmpv\" (UID: \"8d993695-6e46-4b29-98da-6ba5256072a3\") " pod="openshift-console/downloads-85d98c998f-lkmpv"
Sep 07 03:49:20.717290 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.717248    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:20.773592 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.773552    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m84dp\" (UniqueName: \"kubernetes.io/projected/8d993695-6e46-4b29-98da-6ba5256072a3-kube-api-access-m84dp\") pod \"downloads-85d98c998f-lkmpv\" (UID: \"8d993695-6e46-4b29-98da-6ba5256072a3\") " pod="openshift-console/downloads-85d98c998f-lkmpv"
Sep 07 03:49:20.776225 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.776179    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rdgxm"]
Sep 07 03:49:20.784106 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.784081    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m84dp\" (UniqueName: \"kubernetes.io/projected/8d993695-6e46-4b29-98da-6ba5256072a3-kube-api-access-m84dp\") pod \"downloads-85d98c998f-lkmpv\" (UID: \"8d993695-6e46-4b29-98da-6ba5256072a3\") " pod="openshift-console/downloads-85d98c998f-lkmpv"
Sep 07 03:49:20.793925 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.793893    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gcl7p" event={"ID":"f35afb94-a0dc-4039-8d88-81f6fc8326c0","Type":"ContainerStarted","Data":"9b7e0589708c8867e32173c4139f1c06f8c1b4ee17b91e6dcce10103f7096edc"}
Sep 07 03:49:20.814908 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.814851    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gcl7p" podStartSLOduration=2.897875258 podStartE2EDuration="5.814832289s" totalImagesPullingTime="2.916957031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:49:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:49:20.813404699 +0000 UTC m=+47.746885417" watchObservedRunningTime="2026-09-07 03:49:20.814832289 +0000 UTC m=+47.748313008"
Sep 07 03:49:20.850294 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.850262    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:20.874502 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:20.874466    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-lkmpv"
Sep 07 03:49:21.220600 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:21.220571    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:21.304024 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:21.303952    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-lkmpv"]
Sep 07 03:49:21.308105 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:21.308081    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:21.376429 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:49:21.376395    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d993695_6e46_4b29_98da_6ba5256072a3.slice/crio-25f987b45cf277983f8e2a923a499eaf90eb2dade2ce226067cc723e33d11e31 WatchSource:0}: Error finding container 25f987b45cf277983f8e2a923a499eaf90eb2dade2ce226067cc723e33d11e31: Status 404 returned error can't find the container with id 25f987b45cf277983f8e2a923a499eaf90eb2dade2ce226067cc723e33d11e31
Sep 07 03:49:21.614250 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:21.614152    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:21.715194 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:21.715160    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:21.800230 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:21.800181    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nmxn8" event={"ID":"1a092764-0c63-4c0b-8456-a05adb098222","Type":"ContainerStarted","Data":"930ed7160bf0fbe3298b8d8dd2032fa740c6308e8b139818b8e59aaf04700d90"}
Sep 07 03:49:21.801757 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:21.801729    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-lkmpv" event={"ID":"8d993695-6e46-4b29-98da-6ba5256072a3","Type":"ContainerStarted","Data":"25f987b45cf277983f8e2a923a499eaf90eb2dade2ce226067cc723e33d11e31"}
Sep 07 03:49:21.802998 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:21.802934    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rdgxm" event={"ID":"fe79a6a4-0529-4ff4-aabd-7c0a152f2878","Type":"ContainerStarted","Data":"cc156b7658f9f1577e488376d64085913a9367b918df081452160fd50eaa0166"}
Sep 07 03:49:21.803130 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:21.803114    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gcl7p"
Sep 07 03:49:22.053769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:22.053736    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:22.111448 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:22.111381    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:22.231756 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:22.231718    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:22.431027 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:22.430952    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:22.810630 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:22.810521    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rdgxm" event={"ID":"fe79a6a4-0529-4ff4-aabd-7c0a152f2878","Type":"ContainerStarted","Data":"32e8a7eb585a68aa24f29566996482b28507fdff8b77fe2645a9090780593cd5"}
Sep 07 03:49:22.811098 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:22.811081    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rdgxm"
Sep 07 03:49:22.817158 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:22.817133    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rdgxm"
Sep 07 03:49:22.828598 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:22.828545    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-rdgxm" podStartSLOduration=1.355791476 podStartE2EDuration="2.828526518s" totalImagesPullingTime="1.472735042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:49:22.828032076 +0000 UTC m=+49.761512794" watchObservedRunningTime="2026-09-07 03:49:22.828526518 +0000 UTC m=+49.762007541"
Sep 07 03:49:22.829148 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:22.829006    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nmxn8" podStartSLOduration=3.294186384 podStartE2EDuration="7.828992274s" totalImagesPullingTime="4.53480589s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 03:49:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:49:21.817154661 +0000 UTC m=+48.750635378" watchObservedRunningTime="2026-09-07 03:49:22.828992274 +0000 UTC m=+49.762472991"
Sep 07 03:49:22.995629 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:22.995597    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:23.182713 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:23.182628    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:23.244461 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:23.244426    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-5gzmw"]
Sep 07 03:49:23.254073 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:23.254042    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-5gzmw"
Sep 07 03:49:23.256656 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:23.256604    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 07 03:49:23.256806 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:23.256677    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-s7pmh\""
Sep 07 03:49:23.256923 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:23.256908    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 07 03:49:23.257941 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:23.257917    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 07 03:49:23.258168 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:23.258152    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 07 03:49:23.258386 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:23.258371    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 07 03:49:23.259054 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:23.259026    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-5gzmw"]
Sep 07 03:49:23.277020 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:23.276976    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:23.397239 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:23.397184    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/af095918-3620-4785-b968-f2121d20d1db-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-5gzmw\" (UID: \"af095918-3620-4785-b968-f2121d20d1db\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5gzmw"
Sep 07 03:49:23.397434 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:23.397262    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af095918-3620-4785-b968-f2121d20d1db-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-5gzmw\" (UID: \"af095918-3620-4785-b968-f2121d20d1db\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5gzmw"
Sep 07 03:49:23.397434 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:23.397291    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j77wf\" (UniqueName: \"kubernetes.io/projected/af095918-3620-4785-b968-f2121d20d1db-kube-api-access-j77wf\") pod \"prometheus-operator-6f4946db56-5gzmw\" (UID: \"af095918-3620-4785-b968-f2121d20d1db\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5gzmw"
Sep 07 03:49:23.397434 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:23.397330    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af095918-3620-4785-b968-f2121d20d1db-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-5gzmw\" (UID: \"af095918-3620-4785-b968-f2121d20d1db\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5gzmw"
Sep 07 03:49:23.498002 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:23.497955    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/af095918-3620-4785-b968-f2121d20d1db-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-5gzmw\" (UID: \"af095918-3620-4785-b968-f2121d20d1db\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5gzmw"
Sep 07 03:49:23.498183 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:23.498028    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af095918-3620-4785-b968-f2121d20d1db-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-5gzmw\" (UID: \"af095918-3620-4785-b968-f2121d20d1db\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5gzmw"
Sep 07 03:49:23.498183 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:23.498057    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j77wf\" (UniqueName: \"kubernetes.io/projected/af095918-3620-4785-b968-f2121d20d1db-kube-api-access-j77wf\") pod \"prometheus-operator-6f4946db56-5gzmw\" (UID: \"af095918-3620-4785-b968-f2121d20d1db\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5gzmw"
Sep 07 03:49:23.498183 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:23.498121    2583 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 07 03:49:23.498385 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:23.498199    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af095918-3620-4785-b968-f2121d20d1db-prometheus-operator-tls podName:af095918-3620-4785-b968-f2121d20d1db nodeName:}" failed. No retries permitted until 2026-09-07 03:49:23.998177616 +0000 UTC m=+50.931658314 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/af095918-3620-4785-b968-f2121d20d1db-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-5gzmw" (UID: "af095918-3620-4785-b968-f2121d20d1db") : secret "prometheus-operator-tls" not found
Sep 07 03:49:23.498385 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:23.498242    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af095918-3620-4785-b968-f2121d20d1db-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-5gzmw\" (UID: \"af095918-3620-4785-b968-f2121d20d1db\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5gzmw"
Sep 07 03:49:23.499033 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:23.498844    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af095918-3620-4785-b968-f2121d20d1db-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-5gzmw\" (UID: \"af095918-3620-4785-b968-f2121d20d1db\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5gzmw"
Sep 07 03:49:23.502001 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:23.501977    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af095918-3620-4785-b968-f2121d20d1db-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-5gzmw\" (UID: \"af095918-3620-4785-b968-f2121d20d1db\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5gzmw"
Sep 07 03:49:23.507056 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:23.507028    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j77wf\" (UniqueName: \"kubernetes.io/projected/af095918-3620-4785-b968-f2121d20d1db-kube-api-access-j77wf\") pod \"prometheus-operator-6f4946db56-5gzmw\" (UID: \"af095918-3620-4785-b968-f2121d20d1db\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5gzmw"
Sep 07 03:49:23.642369 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:23.642329    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:24.002574 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:24.002539    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/af095918-3620-4785-b968-f2121d20d1db-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-5gzmw\" (UID: \"af095918-3620-4785-b968-f2121d20d1db\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5gzmw"
Sep 07 03:49:24.005700 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:24.005674    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/af095918-3620-4785-b968-f2121d20d1db-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-5gzmw\" (UID: \"af095918-3620-4785-b968-f2121d20d1db\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-5gzmw"
Sep 07 03:49:24.166269 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:24.166232    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-5gzmw"
Sep 07 03:49:24.468152 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:24.468116    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-5gzmw"]
Sep 07 03:49:24.473583 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:49:24.473549    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf095918_3620_4785_b968_f2121d20d1db.slice/crio-613d72c095f34601fa7e704a4e5016f50921bf13ccec06ab035f59e0bc6275be WatchSource:0}: Error finding container 613d72c095f34601fa7e704a4e5016f50921bf13ccec06ab035f59e0bc6275be: Status 404 returned error can't find the container with id 613d72c095f34601fa7e704a4e5016f50921bf13ccec06ab035f59e0bc6275be
Sep 07 03:49:24.693040 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:24.692999    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:24.752085 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:24.751999    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:24.818545 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:24.818463    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-5gzmw" event={"ID":"af095918-3620-4785-b968-f2121d20d1db","Type":"ContainerStarted","Data":"613d72c095f34601fa7e704a4e5016f50921bf13ccec06ab035f59e0bc6275be"}
Sep 07 03:49:25.824937 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:25.824581    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-5gzmw" event={"ID":"af095918-3620-4785-b968-f2121d20d1db","Type":"ContainerStarted","Data":"8a80b779eccc357a3783fef37b0686c2976b53c43c65d55d5491eb0d24fcb148"}
Sep 07 03:49:25.824937 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:25.824657    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-5gzmw" event={"ID":"af095918-3620-4785-b968-f2121d20d1db","Type":"ContainerStarted","Data":"ad387865e21446c30df09ea790ed5141dcf7fb789a566ac95e22b71bf3f10771"}
Sep 07 03:49:25.840388 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:25.840328    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-5gzmw" podStartSLOduration=1.681300114 podStartE2EDuration="2.840308225s" totalImagesPullingTime="1.159008111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:49:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:49:25.839933631 +0000 UTC m=+52.773414361" watchObservedRunningTime="2026-09-07 03:49:25.840308225 +0000 UTC m=+52.773788940"
Sep 07 03:49:25.935622 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:25.935584    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:27.353632 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.353396    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:27.477406 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.477369    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:27.591682 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.591646    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:27.625691 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.625596    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-rwl8n"]
Sep 07 03:49:27.628185 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.628158    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-rwl8n"
Sep 07 03:49:27.632013 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.631795    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 07 03:49:27.632675 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.632646    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-l5b2p\""
Sep 07 03:49:27.632801 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.632693    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 07 03:49:27.645412 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.645383    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-rwl8n"]
Sep 07 03:49:27.665185 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.665153    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"]
Sep 07 03:49:27.667232 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.667193    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"
Sep 07 03:49:27.671009 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.670987    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-fjvh5\""
Sep 07 03:49:27.671512 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.671487    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 07 03:49:27.672177 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.672156    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 07 03:49:27.672688 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.672189    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 07 03:49:27.673575 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.673554    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8r464"]
Sep 07 03:49:27.676420 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.676079    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.678785 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.678759    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 07 03:49:27.678902 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.678805    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qtb9h\""
Sep 07 03:49:27.678902 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.678830    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 07 03:49:27.679190 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.679175    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 07 03:49:27.691077 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.691050    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"]
Sep 07 03:49:27.697513 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.697487    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:27.733271 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.733234    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73a4703b-3eaf-4d46-a94f-c45676a063f3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-rwl8n\" (UID: \"73a4703b-3eaf-4d46-a94f-c45676a063f3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-rwl8n"
Sep 07 03:49:27.733462 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.733363    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73a4703b-3eaf-4d46-a94f-c45676a063f3-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-rwl8n\" (UID: \"73a4703b-3eaf-4d46-a94f-c45676a063f3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-rwl8n"
Sep 07 03:49:27.733462 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.733404    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckggf\" (UniqueName: \"kubernetes.io/projected/73a4703b-3eaf-4d46-a94f-c45676a063f3-kube-api-access-ckggf\") pod \"openshift-state-metrics-657fd6c54-rwl8n\" (UID: \"73a4703b-3eaf-4d46-a94f-c45676a063f3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-rwl8n"
Sep 07 03:49:27.733550 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.733484    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/73a4703b-3eaf-4d46-a94f-c45676a063f3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-rwl8n\" (UID: \"73a4703b-3eaf-4d46-a94f-c45676a063f3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-rwl8n"
Sep 07 03:49:27.795899 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.795867    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:27.834451 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.834413    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73a4703b-3eaf-4d46-a94f-c45676a063f3-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-rwl8n\" (UID: \"73a4703b-3eaf-4d46-a94f-c45676a063f3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-rwl8n"
Sep 07 03:49:27.834639 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.834465    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqvgf\" (UniqueName: \"kubernetes.io/projected/211cb420-6847-48a1-8067-7a0ef7238716-kube-api-access-kqvgf\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.834639 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.834497    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/73a4703b-3eaf-4d46-a94f-c45676a063f3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-rwl8n\" (UID: \"73a4703b-3eaf-4d46-a94f-c45676a063f3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-rwl8n"
Sep 07 03:49:27.834639 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.834551    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0500a2e-c89d-4b11-b01f-050faaa07bc5-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-t6fnv\" (UID: \"a0500a2e-c89d-4b11-b01f-050faaa07bc5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"
Sep 07 03:49:27.834639 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.834580    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/211cb420-6847-48a1-8067-7a0ef7238716-node-exporter-textfile\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.834639 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.834608    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a0500a2e-c89d-4b11-b01f-050faaa07bc5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-t6fnv\" (UID: \"a0500a2e-c89d-4b11-b01f-050faaa07bc5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"
Sep 07 03:49:27.834639 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.834637    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/211cb420-6847-48a1-8067-7a0ef7238716-node-exporter-wtmp\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.834858 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.834663    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/211cb420-6847-48a1-8067-7a0ef7238716-sys\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.834858 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.834684    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm9h4\" (UniqueName: \"kubernetes.io/projected/a0500a2e-c89d-4b11-b01f-050faaa07bc5-kube-api-access-pm9h4\") pod \"kube-state-metrics-74cb57df7-t6fnv\" (UID: \"a0500a2e-c89d-4b11-b01f-050faaa07bc5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"
Sep 07 03:49:27.834858 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.834709    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckggf\" (UniqueName: \"kubernetes.io/projected/73a4703b-3eaf-4d46-a94f-c45676a063f3-kube-api-access-ckggf\") pod \"openshift-state-metrics-657fd6c54-rwl8n\" (UID: \"73a4703b-3eaf-4d46-a94f-c45676a063f3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-rwl8n"
Sep 07 03:49:27.834858 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.834724    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/211cb420-6847-48a1-8067-7a0ef7238716-root\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.834858 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.834743    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0500a2e-c89d-4b11-b01f-050faaa07bc5-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-t6fnv\" (UID: \"a0500a2e-c89d-4b11-b01f-050faaa07bc5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"
Sep 07 03:49:27.834858 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.834782    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/211cb420-6847-48a1-8067-7a0ef7238716-node-exporter-tls\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.834858 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.834799    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a0500a2e-c89d-4b11-b01f-050faaa07bc5-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-t6fnv\" (UID: \"a0500a2e-c89d-4b11-b01f-050faaa07bc5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"
Sep 07 03:49:27.834858 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.834834    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73a4703b-3eaf-4d46-a94f-c45676a063f3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-rwl8n\" (UID: \"73a4703b-3eaf-4d46-a94f-c45676a063f3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-rwl8n"
Sep 07 03:49:27.834858 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.834858    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/211cb420-6847-48a1-8067-7a0ef7238716-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.835147 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.834876    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0500a2e-c89d-4b11-b01f-050faaa07bc5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-t6fnv\" (UID: \"a0500a2e-c89d-4b11-b01f-050faaa07bc5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"
Sep 07 03:49:27.835147 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.834898    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/211cb420-6847-48a1-8067-7a0ef7238716-node-exporter-accelerators-collector-config\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.835147 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.834916    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/211cb420-6847-48a1-8067-7a0ef7238716-metrics-client-ca\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.835324 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.835177    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73a4703b-3eaf-4d46-a94f-c45676a063f3-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-rwl8n\" (UID: \"73a4703b-3eaf-4d46-a94f-c45676a063f3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-rwl8n"
Sep 07 03:49:27.838904 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.838880    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73a4703b-3eaf-4d46-a94f-c45676a063f3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-rwl8n\" (UID: \"73a4703b-3eaf-4d46-a94f-c45676a063f3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-rwl8n"
Sep 07 03:49:27.838904 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.838892    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/73a4703b-3eaf-4d46-a94f-c45676a063f3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-rwl8n\" (UID: \"73a4703b-3eaf-4d46-a94f-c45676a063f3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-rwl8n"
Sep 07 03:49:27.852446 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.852416    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckggf\" (UniqueName: \"kubernetes.io/projected/73a4703b-3eaf-4d46-a94f-c45676a063f3-kube-api-access-ckggf\") pod \"openshift-state-metrics-657fd6c54-rwl8n\" (UID: \"73a4703b-3eaf-4d46-a94f-c45676a063f3\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-rwl8n"
Sep 07 03:49:27.895735 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.895650    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:27.935942 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.935905    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqvgf\" (UniqueName: \"kubernetes.io/projected/211cb420-6847-48a1-8067-7a0ef7238716-kube-api-access-kqvgf\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.935942 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.935956    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0500a2e-c89d-4b11-b01f-050faaa07bc5-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-t6fnv\" (UID: \"a0500a2e-c89d-4b11-b01f-050faaa07bc5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"
Sep 07 03:49:27.936174 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.935983    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/211cb420-6847-48a1-8067-7a0ef7238716-node-exporter-textfile\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.936174 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.936023    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a0500a2e-c89d-4b11-b01f-050faaa07bc5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-t6fnv\" (UID: \"a0500a2e-c89d-4b11-b01f-050faaa07bc5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"
Sep 07 03:49:27.936174 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.936052    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/211cb420-6847-48a1-8067-7a0ef7238716-node-exporter-wtmp\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.936174 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.936079    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/211cb420-6847-48a1-8067-7a0ef7238716-sys\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.936174 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.936105    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm9h4\" (UniqueName: \"kubernetes.io/projected/a0500a2e-c89d-4b11-b01f-050faaa07bc5-kube-api-access-pm9h4\") pod \"kube-state-metrics-74cb57df7-t6fnv\" (UID: \"a0500a2e-c89d-4b11-b01f-050faaa07bc5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"
Sep 07 03:49:27.936174 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.936133    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/211cb420-6847-48a1-8067-7a0ef7238716-root\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.936174 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.936161    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0500a2e-c89d-4b11-b01f-050faaa07bc5-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-t6fnv\" (UID: \"a0500a2e-c89d-4b11-b01f-050faaa07bc5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"
Sep 07 03:49:27.936536 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.936199    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/211cb420-6847-48a1-8067-7a0ef7238716-node-exporter-tls\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.936536 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.936244    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a0500a2e-c89d-4b11-b01f-050faaa07bc5-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-t6fnv\" (UID: \"a0500a2e-c89d-4b11-b01f-050faaa07bc5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"
Sep 07 03:49:27.936536 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.936295    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/211cb420-6847-48a1-8067-7a0ef7238716-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.936536 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.936323    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0500a2e-c89d-4b11-b01f-050faaa07bc5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-t6fnv\" (UID: \"a0500a2e-c89d-4b11-b01f-050faaa07bc5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"
Sep 07 03:49:27.936536 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.936356    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/211cb420-6847-48a1-8067-7a0ef7238716-node-exporter-accelerators-collector-config\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.936536 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.936386    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/211cb420-6847-48a1-8067-7a0ef7238716-metrics-client-ca\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.936536 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.936420    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/211cb420-6847-48a1-8067-7a0ef7238716-node-exporter-textfile\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.937247 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.936918    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/211cb420-6847-48a1-8067-7a0ef7238716-metrics-client-ca\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.937247 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.937000    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0500a2e-c89d-4b11-b01f-050faaa07bc5-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-t6fnv\" (UID: \"a0500a2e-c89d-4b11-b01f-050faaa07bc5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"
Sep 07 03:49:27.937247 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.937018    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/211cb420-6847-48a1-8067-7a0ef7238716-sys\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.937247 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:27.937063    2583 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 07 03:49:27.937247 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:27.937116    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/211cb420-6847-48a1-8067-7a0ef7238716-node-exporter-tls podName:211cb420-6847-48a1-8067-7a0ef7238716 nodeName:}" failed. No retries permitted until 2026-09-07 03:49:28.437099806 +0000 UTC m=+55.370580504 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/211cb420-6847-48a1-8067-7a0ef7238716-node-exporter-tls") pod "node-exporter-8r464" (UID: "211cb420-6847-48a1-8067-7a0ef7238716") : secret "node-exporter-tls" not found
Sep 07 03:49:27.937247 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.937135    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/211cb420-6847-48a1-8067-7a0ef7238716-node-exporter-wtmp\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.937247 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.937152    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a0500a2e-c89d-4b11-b01f-050faaa07bc5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-t6fnv\" (UID: \"a0500a2e-c89d-4b11-b01f-050faaa07bc5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"
Sep 07 03:49:27.937247 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:27.937189    2583 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 07 03:49:27.937651 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.937331    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a0500a2e-c89d-4b11-b01f-050faaa07bc5-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-t6fnv\" (UID: \"a0500a2e-c89d-4b11-b01f-050faaa07bc5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"
Sep 07 03:49:27.937651 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:49:27.937343    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0500a2e-c89d-4b11-b01f-050faaa07bc5-kube-state-metrics-tls podName:a0500a2e-c89d-4b11-b01f-050faaa07bc5 nodeName:}" failed. No retries permitted until 2026-09-07 03:49:28.437326009 +0000 UTC m=+55.370806707 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a0500a2e-c89d-4b11-b01f-050faaa07bc5-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-t6fnv" (UID: "a0500a2e-c89d-4b11-b01f-050faaa07bc5") : secret "kube-state-metrics-tls" not found
Sep 07 03:49:27.937651 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.937599    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/211cb420-6847-48a1-8067-7a0ef7238716-root\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.937830 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.937808    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/211cb420-6847-48a1-8067-7a0ef7238716-node-exporter-accelerators-collector-config\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.939255 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.939235    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-rwl8n"
Sep 07 03:49:27.939503 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.939465    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/211cb420-6847-48a1-8067-7a0ef7238716-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.939727 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.939709    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0500a2e-c89d-4b11-b01f-050faaa07bc5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-t6fnv\" (UID: \"a0500a2e-c89d-4b11-b01f-050faaa07bc5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"
Sep 07 03:49:27.950389 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.946575    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm9h4\" (UniqueName: \"kubernetes.io/projected/a0500a2e-c89d-4b11-b01f-050faaa07bc5-kube-api-access-pm9h4\") pod \"kube-state-metrics-74cb57df7-t6fnv\" (UID: \"a0500a2e-c89d-4b11-b01f-050faaa07bc5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"
Sep 07 03:49:27.950389 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.947253    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqvgf\" (UniqueName: \"kubernetes.io/projected/211cb420-6847-48a1-8067-7a0ef7238716-kube-api-access-kqvgf\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:27.997522 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:27.997470    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:28.097738 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:28.097701    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:28.197963 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:28.197932    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:28.230616 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:28.230584    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-rwl8n"]
Sep 07 03:49:28.235167 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:49:28.235137    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73a4703b_3eaf_4d46_a94f_c45676a063f3.slice/crio-99751088ed5e2a5215e9dadcf4094108da0d92f177516aa59aa8d4dd2d6d8432 WatchSource:0}: Error finding container 99751088ed5e2a5215e9dadcf4094108da0d92f177516aa59aa8d4dd2d6d8432: Status 404 returned error can't find the container with id 99751088ed5e2a5215e9dadcf4094108da0d92f177516aa59aa8d4dd2d6d8432
Sep 07 03:49:28.298061 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:28.298031    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:28.441152 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:28.441114    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0500a2e-c89d-4b11-b01f-050faaa07bc5-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-t6fnv\" (UID: \"a0500a2e-c89d-4b11-b01f-050faaa07bc5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"
Sep 07 03:49:28.441923 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:28.441163    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/211cb420-6847-48a1-8067-7a0ef7238716-node-exporter-tls\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:28.444022 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:28.443989    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/211cb420-6847-48a1-8067-7a0ef7238716-node-exporter-tls\") pod \"node-exporter-8r464\" (UID: \"211cb420-6847-48a1-8067-7a0ef7238716\") " pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:28.444207 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:28.444186    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0500a2e-c89d-4b11-b01f-050faaa07bc5-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-t6fnv\" (UID: \"a0500a2e-c89d-4b11-b01f-050faaa07bc5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"
Sep 07 03:49:28.578628 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:28.578546    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"
Sep 07 03:49:28.587238 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:28.587189    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8r464"
Sep 07 03:49:28.589597 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:28.589568    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:28.690748 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:28.690721    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:28.794746 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:28.794710    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:28.845723 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:28.845645    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8r464" event={"ID":"211cb420-6847-48a1-8067-7a0ef7238716","Type":"ContainerStarted","Data":"c3de2f5d9fa650800ea85fbc5f82e51863aeb0b84a8ce823ceb15aa87938c45d"}
Sep 07 03:49:28.848081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:28.847944    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-rwl8n" event={"ID":"73a4703b-3eaf-4d46-a94f-c45676a063f3","Type":"ContainerStarted","Data":"b1cd5ccc8d96eef046e33e1fb43b2929b936aee7284e81f2acb6d521c38b5fdf"}
Sep 07 03:49:28.848081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:28.848007    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-rwl8n" event={"ID":"73a4703b-3eaf-4d46-a94f-c45676a063f3","Type":"ContainerStarted","Data":"2730a23031b36112fa822d077de2a98948368952956f56a6847c98e89a1444ad"}
Sep 07 03:49:28.848081 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:28.848038    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-rwl8n" event={"ID":"73a4703b-3eaf-4d46-a94f-c45676a063f3","Type":"ContainerStarted","Data":"99751088ed5e2a5215e9dadcf4094108da0d92f177516aa59aa8d4dd2d6d8432"}
Sep 07 03:49:28.870747 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:28.870716    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv"]
Sep 07 03:49:28.874355 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:49:28.874327    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0500a2e_c89d_4b11_b01f_050faaa07bc5.slice/crio-6c5ba86f5659e830774d219e8dd5120686da68e8a792ae1e2ba5bd857fb98c22 WatchSource:0}: Error finding container 6c5ba86f5659e830774d219e8dd5120686da68e8a792ae1e2ba5bd857fb98c22: Status 404 returned error can't find the container with id 6c5ba86f5659e830774d219e8dd5120686da68e8a792ae1e2ba5bd857fb98c22
Sep 07 03:49:28.882941 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:28.882910    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:28.992685 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:28.992645    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:29.663853 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.663826    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"]
Sep 07 03:49:29.671879 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.671855    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.677249 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.677207    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-6d7vp\""
Sep 07 03:49:29.677923 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.677657    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 07 03:49:29.677923 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.677810    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 07 03:49:29.678101 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.677951    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 07 03:49:29.678260 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.678241    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 07 03:49:29.678364 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.678348    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 07 03:49:29.684892 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.684683    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-ab25or4f0vsju\""
Sep 07 03:49:29.690491 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.690373    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"]
Sep 07 03:49:29.853033 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.852998    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0372d650-50ef-4bc7-a9e7-dd092bb32204-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.853228 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.853055    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0372d650-50ef-4bc7-a9e7-dd092bb32204-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.853228 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.853102    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0372d650-50ef-4bc7-a9e7-dd092bb32204-metrics-client-ca\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.853228 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.853136    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdpmr\" (UniqueName: \"kubernetes.io/projected/0372d650-50ef-4bc7-a9e7-dd092bb32204-kube-api-access-mdpmr\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.853228 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.853198    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0372d650-50ef-4bc7-a9e7-dd092bb32204-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.853228 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.853205    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv" event={"ID":"a0500a2e-c89d-4b11-b01f-050faaa07bc5","Type":"ContainerStarted","Data":"6c5ba86f5659e830774d219e8dd5120686da68e8a792ae1e2ba5bd857fb98c22"}
Sep 07 03:49:29.853519 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.853249    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0372d650-50ef-4bc7-a9e7-dd092bb32204-secret-grpc-tls\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.853519 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.853339    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0372d650-50ef-4bc7-a9e7-dd092bb32204-secret-thanos-querier-tls\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.853519 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.853367    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0372d650-50ef-4bc7-a9e7-dd092bb32204-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.855266 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.855235    2583 generic.go:356] "Generic (PLEG): container finished" podID="211cb420-6847-48a1-8067-7a0ef7238716" containerID="45dc594f046948e4a8ea468f3ab91bd62f9fe08cda7d5b01fb0dcaa2f0e8961a" exitCode=0
Sep 07 03:49:29.855390 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.855279    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8r464" event={"ID":"211cb420-6847-48a1-8067-7a0ef7238716","Type":"ContainerDied","Data":"45dc594f046948e4a8ea468f3ab91bd62f9fe08cda7d5b01fb0dcaa2f0e8961a"}
Sep 07 03:49:29.857820 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.857797    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-rwl8n" event={"ID":"73a4703b-3eaf-4d46-a94f-c45676a063f3","Type":"ContainerStarted","Data":"6a6f59235f1cbdd18cf5f675a3d01d55fc9e20514030593aecd89c48623bd1cf"}
Sep 07 03:49:29.908440 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.906968    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-rwl8n" podStartSLOduration=1.67787277 podStartE2EDuration="2.906945943s" totalImagesPullingTime="1.229073173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:49:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:49:29.906485649 +0000 UTC m=+56.839966366" watchObservedRunningTime="2026-09-07 03:49:29.906945943 +0000 UTC m=+56.840426664"
Sep 07 03:49:29.954496 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.954463    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0372d650-50ef-4bc7-a9e7-dd092bb32204-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.954642 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.954502    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0372d650-50ef-4bc7-a9e7-dd092bb32204-metrics-client-ca\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.954642 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.954533    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdpmr\" (UniqueName: \"kubernetes.io/projected/0372d650-50ef-4bc7-a9e7-dd092bb32204-kube-api-access-mdpmr\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.954642 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.954629    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0372d650-50ef-4bc7-a9e7-dd092bb32204-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.954811 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.954657    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0372d650-50ef-4bc7-a9e7-dd092bb32204-secret-grpc-tls\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.954811 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.954753    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0372d650-50ef-4bc7-a9e7-dd092bb32204-secret-thanos-querier-tls\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.954890 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.954805    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0372d650-50ef-4bc7-a9e7-dd092bb32204-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.954923 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.954888    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0372d650-50ef-4bc7-a9e7-dd092bb32204-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.955915 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.955572    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0372d650-50ef-4bc7-a9e7-dd092bb32204-metrics-client-ca\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.959138 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.959053    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0372d650-50ef-4bc7-a9e7-dd092bb32204-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.959283 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.959170    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0372d650-50ef-4bc7-a9e7-dd092bb32204-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.959775 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.959743    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0372d650-50ef-4bc7-a9e7-dd092bb32204-secret-grpc-tls\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.960076 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.960055    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0372d650-50ef-4bc7-a9e7-dd092bb32204-secret-thanos-querier-tls\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.960383 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.960362    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0372d650-50ef-4bc7-a9e7-dd092bb32204-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.961422 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.961394    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0372d650-50ef-4bc7-a9e7-dd092bb32204-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:29.964767 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:29.964748    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdpmr\" (UniqueName: \"kubernetes.io/projected/0372d650-50ef-4bc7-a9e7-dd092bb32204-kube-api-access-mdpmr\") pod \"thanos-querier-5765d7cf8b-pjfrx\" (UID: \"0372d650-50ef-4bc7-a9e7-dd092bb32204\") " pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:30.000985 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:30.000953    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:30.589682 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:30.589653    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:30.691598 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:30.691551    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:30.708907 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:30.708883    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"]
Sep 07 03:49:30.716085 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:49:30.716059    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0372d650_50ef_4bc7_a9e7_dd092bb32204.slice/crio-8f8550134f3e9dc2dfd671a352d9b273dd8af47e5add4102995a411237746036 WatchSource:0}: Error finding container 8f8550134f3e9dc2dfd671a352d9b273dd8af47e5add4102995a411237746036: Status 404 returned error can't find the container with id 8f8550134f3e9dc2dfd671a352d9b273dd8af47e5add4102995a411237746036
Sep 07 03:49:30.865072 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:30.865035    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv" event={"ID":"a0500a2e-c89d-4b11-b01f-050faaa07bc5","Type":"ContainerStarted","Data":"f5f4692f51050a18687f9a00a381f057c7de87e5eb25ad9f299f26f8230c434b"}
Sep 07 03:49:30.866550 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:30.866517    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx" event={"ID":"0372d650-50ef-4bc7-a9e7-dd092bb32204","Type":"ContainerStarted","Data":"8f8550134f3e9dc2dfd671a352d9b273dd8af47e5add4102995a411237746036"}
Sep 07 03:49:30.869523 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:30.869493    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8r464" event={"ID":"211cb420-6847-48a1-8067-7a0ef7238716","Type":"ContainerStarted","Data":"955b357ed6646174254a0277d3f4916d113005a1516024731d1d6e035b1b7391"}
Sep 07 03:49:30.869685 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:30.869552    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8r464" event={"ID":"211cb420-6847-48a1-8067-7a0ef7238716","Type":"ContainerStarted","Data":"009a0514bba15d172038210fa0634191197471c5cc3005eccc40e2d8776525ed"}
Sep 07 03:49:30.889754 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:30.889705    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8r464" podStartSLOduration=2.787159822 podStartE2EDuration="3.889687013s" totalImagesPullingTime="1.023475596s" totalInitContainerRuntime="79.051595ms" isStatefulPod=true podCreationTimestamp="2026-09-07 03:49:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:49:30.88912246 +0000 UTC m=+57.822603180" watchObservedRunningTime="2026-09-07 03:49:30.889687013 +0000 UTC m=+57.823167730"
Sep 07 03:49:30.901502 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:30.901434    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:30.999964 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:30.999772    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:31.655293 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:31.655260    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:31.733354 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:31.733318    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:31.814935 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:31.814909    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gcl7p"
Sep 07 03:49:31.835872 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:31.835821    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:32.376334 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:32.376305    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:32.475562 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:32.475521    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:32.604753 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:32.604710    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:33.258250 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.258195    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:33.360347 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.360287    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:33.875817 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.875779    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:33.882907 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.882881    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 03:49:33.888967 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.888938    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:33.892245 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.892191    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 07 03:49:33.892781 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.892762    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 07 03:49:33.892973 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.892958    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-w62b8\""
Sep 07 03:49:33.894010 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.892229    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 07 03:49:33.894010 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.893579    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 07 03:49:33.894010 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.893720    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 07 03:49:33.894010 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.893841    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 07 03:49:33.894010 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.893847    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 07 03:49:33.894010 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.893995    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 07 03:49:33.894365 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.894302    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-72t2596m39tvm\""
Sep 07 03:49:33.895881 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.895856    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 07 03:49:33.896119 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.896100    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 07 03:49:33.897373 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.897155    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 07 03:49:33.897373 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.897307    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 07 03:49:33.898124 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.897542    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 07 03:49:33.899155 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.898677    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 07 03:49:33.905632 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.905608    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 07 03:49:33.914384 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.914273    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 03:49:33.986316 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.986160    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:33.986316 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.986238    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:33.986316 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.986270    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:33.986316 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.986285    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:33.986316 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.986310    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:33.986755 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.986365    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-web-config\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:33.986755 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.986399    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:33.986755 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.986424    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:33.986755 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.986465    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:33.986755 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.986490    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:33.986755 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.986509    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:33.986755 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.986544    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:33.986755 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.986577    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:33.986755 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.986618    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:33.986755 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.986693    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/021172f4-4a0f-4980-8c47-1438394eec8d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:33.986755 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.986738    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:33.987350 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.986825    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/021172f4-4a0f-4980-8c47-1438394eec8d-config-out\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:33.987350 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.986846    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:33.987350 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.986862    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-config\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:33.987350 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:33.986881    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bcwp\" (UniqueName: \"kubernetes.io/projected/021172f4-4a0f-4980-8c47-1438394eec8d-kube-api-access-2bcwp\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.088713 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.087956    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/021172f4-4a0f-4980-8c47-1438394eec8d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.088713 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.088005    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.088713 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.088053    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/021172f4-4a0f-4980-8c47-1438394eec8d-config-out\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.088713 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.088071    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.088713 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.088095    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-config\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.088713 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.088117    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bcwp\" (UniqueName: \"kubernetes.io/projected/021172f4-4a0f-4980-8c47-1438394eec8d-kube-api-access-2bcwp\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.088713 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.088154    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.088713 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.088181    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.088713 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.088204    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.088713 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.088238    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.088713 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.088269    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.088713 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.088307    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-web-config\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.088713 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.088333    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.088713 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.088362    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.088713 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.088388    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.088713 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.088407    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.089796 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.089116    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.089796 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.089157    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.089796 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.089207    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.089796 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.089299    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.089796 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.089327    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.089796 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.089352    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.089796 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.089372    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.091173 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.091150    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/021172f4-4a0f-4980-8c47-1438394eec8d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.091795 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.091764    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.092386 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.092359    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.093881 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.093544    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.094240 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.094040    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/021172f4-4a0f-4980-8c47-1438394eec8d-config-out\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.095063 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.094767    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.095063 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.095055    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-web-config\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.096537 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.096512    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.096759 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.096734    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.097593 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.097567    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.097731 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.097707    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-config\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.098753 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.098707    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.100512 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.100474    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.101977 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.101953    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.102324 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.102305    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.103686 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.103650    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.106204 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.106184    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bcwp\" (UniqueName: \"kubernetes.io/projected/021172f4-4a0f-4980-8c47-1438394eec8d-kube-api-access-2bcwp\") pod \"prometheus-k8s-0\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.205453 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.205409    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:49:34.744917 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.744885    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nk8kh"
Sep 07 03:49:34.877109 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.877067    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:34.944619 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:34.944578    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:36.605699 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:36.605647    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:36.667666 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:36.667626    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:37.256587 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:37.256553    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:38.329983 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:38.329953    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:38.430223 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:38.430184    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:38.602280 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:38.598854    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 03:49:38.914855 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:38.913829    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv" event={"ID":"a0500a2e-c89d-4b11-b01f-050faaa07bc5","Type":"ContainerStarted","Data":"9d1e4033cac6eb031eb33e2e96788278f15f53577ca1102a06aba8f44be9e787"}
Sep 07 03:49:38.914855 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:38.913909    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv" event={"ID":"a0500a2e-c89d-4b11-b01f-050faaa07bc5","Type":"ContainerStarted","Data":"3fc85d2baa57a5dd8a82f74208ba419c18ab89ded07850915202cf55cd604f00"}
Sep 07 03:49:38.916737 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:38.916633    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"021172f4-4a0f-4980-8c47-1438394eec8d","Type":"ContainerStarted","Data":"5105a3501610a0cdc8af38d1483bf70fab1b46327163194f64aa3817f523a379"}
Sep 07 03:49:38.921250 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:38.920451    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-lkmpv" event={"ID":"8d993695-6e46-4b29-98da-6ba5256072a3","Type":"ContainerStarted","Data":"369f1a342923f078db975a0ee709afe8598578b1e0460870564aa25c49fc9c95"}
Sep 07 03:49:38.921250 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:38.921168    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-lkmpv"
Sep 07 03:49:38.922467 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:38.922433    2583 patch_prober.go:28] interesting pod/downloads-85d98c998f-lkmpv container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.10:8080/\": dial tcp 10.134.0.10:8080: connect: connection refused" start-of-body=
Sep 07 03:49:38.922563 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:38.922494    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-85d98c998f-lkmpv" podUID="8d993695-6e46-4b29-98da-6ba5256072a3" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.10:8080/\": dial tcp 10.134.0.10:8080: connect: connection refused"
Sep 07 03:49:38.933655 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:38.933064    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-t6fnv" podStartSLOduration=10.070606098 podStartE2EDuration="11.933051159s" totalImagesPullingTime="1.862445061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:49:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:49:38.932105767 +0000 UTC m=+65.865586521" watchObservedRunningTime="2026-09-07 03:49:38.933051159 +0000 UTC m=+65.866531911"
Sep 07 03:49:38.950504 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:38.950432    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-lkmpv" podStartSLOduration=1.7247464479999999 podStartE2EDuration="18.950410958s" totalImagesPullingTime="17.22566451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:49:38.949076324 +0000 UTC m=+65.882557097" watchObservedRunningTime="2026-09-07 03:49:38.950410958 +0000 UTC m=+65.883891677"
Sep 07 03:49:39.237694 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:39.237136    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs\") pod \"network-metrics-daemon-sml8b\" (UID: \"bb13bb24-ca38-4d13-aa0c-1c47eb028ae1\") " pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:49:39.240439 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:39.240409    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 07 03:49:39.264555 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:39.264478    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb13bb24-ca38-4d13-aa0c-1c47eb028ae1-metrics-certs\") pod \"network-metrics-daemon-sml8b\" (UID: \"bb13bb24-ca38-4d13-aa0c-1c47eb028ae1\") " pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:49:39.328014 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:39.327981    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-25ldn\""
Sep 07 03:49:39.335664 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:39.335399    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sml8b"
Sep 07 03:49:39.337889 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:39.337741    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret\") pod \"global-pull-secret-syncer-v9fzl\" (UID: \"244e7a50-68b4-41dc-aa4b-8b83813cf489\") " pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:49:39.340618 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:39.340599    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 07 03:49:39.353708 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:39.353679    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/244e7a50-68b4-41dc-aa4b-8b83813cf489-original-pull-secret\") pod \"global-pull-secret-syncer-v9fzl\" (UID: \"244e7a50-68b4-41dc-aa4b-8b83813cf489\") " pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:49:39.438678 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:39.438390    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfcn7\" (UniqueName: \"kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7\") pod \"network-check-target-qbsbx\" (UID: \"4cf74d00-0e92-415e-8129-2033297d3707\") " pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:49:39.441113 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:39.440850    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v9fzl"
Sep 07 03:49:39.442519 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:39.441582    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 07 03:49:39.451794 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:39.451579    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 07 03:49:39.463466 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:39.463344    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfcn7\" (UniqueName: \"kubernetes.io/projected/4cf74d00-0e92-415e-8129-2033297d3707-kube-api-access-kfcn7\") pod \"network-check-target-qbsbx\" (UID: \"4cf74d00-0e92-415e-8129-2033297d3707\") " pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:49:39.650710 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:39.650622    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sml8b"]
Sep 07 03:49:39.735428 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:39.735391    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-v9fzl"]
Sep 07 03:49:39.738099 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:39.737916    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wrs9m\""
Sep 07 03:49:39.746882 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:39.746637    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:49:39.938633 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:39.938402    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-lkmpv"
Sep 07 03:49:40.642900 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:40.642830    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:40.827957 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:49:40.827917    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb13bb24_ca38_4d13_aa0c_1c47eb028ae1.slice/crio-6b1db050750a67cf7f25ab7515cb1dea4fbac080b4d131aa30e716b41d198a07 WatchSource:0}: Error finding container 6b1db050750a67cf7f25ab7515cb1dea4fbac080b4d131aa30e716b41d198a07: Status 404 returned error can't find the container with id 6b1db050750a67cf7f25ab7515cb1dea4fbac080b4d131aa30e716b41d198a07
Sep 07 03:49:40.829137 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:49:40.828946    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod244e7a50_68b4_41dc_aa4b_8b83813cf489.slice/crio-869970b9c36ebfa35ddf0d2a86a7704302ec39552509fe59c07e59fc6c6dcc95 WatchSource:0}: Error finding container 869970b9c36ebfa35ddf0d2a86a7704302ec39552509fe59c07e59fc6c6dcc95: Status 404 returned error can't find the container with id 869970b9c36ebfa35ddf0d2a86a7704302ec39552509fe59c07e59fc6c6dcc95
Sep 07 03:49:40.930641 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:40.930555    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-v9fzl" event={"ID":"244e7a50-68b4-41dc-aa4b-8b83813cf489","Type":"ContainerStarted","Data":"869970b9c36ebfa35ddf0d2a86a7704302ec39552509fe59c07e59fc6c6dcc95"}
Sep 07 03:49:40.932103 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:40.932063    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sml8b" event={"ID":"bb13bb24-ca38-4d13-aa0c-1c47eb028ae1","Type":"ContainerStarted","Data":"6b1db050750a67cf7f25ab7515cb1dea4fbac080b4d131aa30e716b41d198a07"}
Sep 07 03:49:41.132594 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:41.132561    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:41.189743 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:41.189707    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:41.613272 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:41.613235    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qbsbx"]
Sep 07 03:49:41.617077 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:49:41.617051    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cf74d00_0e92_415e_8129_2033297d3707.slice/crio-f87a7df759d488d771cb9f303b2459f48dadd6dfc5c00b1787d0ba7473758333 WatchSource:0}: Error finding container f87a7df759d488d771cb9f303b2459f48dadd6dfc5c00b1787d0ba7473758333: Status 404 returned error can't find the container with id f87a7df759d488d771cb9f303b2459f48dadd6dfc5c00b1787d0ba7473758333
Sep 07 03:49:41.941559 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:41.941505    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qbsbx" event={"ID":"4cf74d00-0e92-415e-8129-2033297d3707","Type":"ContainerStarted","Data":"f87a7df759d488d771cb9f303b2459f48dadd6dfc5c00b1787d0ba7473758333"}
Sep 07 03:49:41.945543 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:41.945499    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx" event={"ID":"0372d650-50ef-4bc7-a9e7-dd092bb32204","Type":"ContainerStarted","Data":"4347d1746738abd08202296b497de51dcfb656b0ae97d08fa688f3c80e454090"}
Sep 07 03:49:41.945689 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:41.945582    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx" event={"ID":"0372d650-50ef-4bc7-a9e7-dd092bb32204","Type":"ContainerStarted","Data":"59d35e59113815e931d877b3e5b5fe0985171f65d0d227d88a5f55a3d8c15a9e"}
Sep 07 03:49:41.945689 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:41.945624    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx" event={"ID":"0372d650-50ef-4bc7-a9e7-dd092bb32204","Type":"ContainerStarted","Data":"6a6a9fa89e20f07bc98cf65edcb737530fe2352fc111f4006a3b5fbf9f08d405"}
Sep 07 03:49:41.948307 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:41.948275    2583 generic.go:356] "Generic (PLEG): container finished" podID="021172f4-4a0f-4980-8c47-1438394eec8d" containerID="6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802" exitCode=0
Sep 07 03:49:41.948541 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:41.948519    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"021172f4-4a0f-4980-8c47-1438394eec8d","Type":"ContainerDied","Data":"6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802"}
Sep 07 03:49:44.974072 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:44.974028    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx" event={"ID":"0372d650-50ef-4bc7-a9e7-dd092bb32204","Type":"ContainerStarted","Data":"243bc6bed6b5508e97c2841cd770c57e5363bb82cde2caa67ab642bc2f74167a"}
Sep 07 03:49:44.974642 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:44.974157    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx" event={"ID":"0372d650-50ef-4bc7-a9e7-dd092bb32204","Type":"ContainerStarted","Data":"4bb959285c92e01bccfcac642690b4556a27aeb4f852cae972ead371ab7c5a11"}
Sep 07 03:49:44.974642 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:44.974205    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx" event={"ID":"0372d650-50ef-4bc7-a9e7-dd092bb32204","Type":"ContainerStarted","Data":"43e479da178f45c363f513bdddf69ab934bf44c0ec64eb8c98dfdc92eb3048d4"}
Sep 07 03:49:44.975383 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:44.975362    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:44.978277 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:44.978247    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sml8b" event={"ID":"bb13bb24-ca38-4d13-aa0c-1c47eb028ae1","Type":"ContainerStarted","Data":"0d2c456e8df4f38eff85ae12b31da2433201e882c7976ef320ff82060b2b53a9"}
Sep 07 03:49:44.978398 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:44.978307    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sml8b" event={"ID":"bb13bb24-ca38-4d13-aa0c-1c47eb028ae1","Type":"ContainerStarted","Data":"e4ae34b4c33a021e776e0f51a066df91454806e07b029c1490d568b5a2625522"}
Sep 07 03:49:45.030531 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:45.027666    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx" podStartSLOduration=3.335410198 podStartE2EDuration="16.027643524s" totalImagesPullingTime="12.692233326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:49:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:49:45.00674033 +0000 UTC m=+71.940221048" watchObservedRunningTime="2026-09-07 03:49:45.027643524 +0000 UTC m=+71.961124243"
Sep 07 03:49:45.030531 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:45.028786    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sml8b" podStartSLOduration=68.326656489 podStartE2EDuration="1m11.028766979s" totalImagesPullingTime="2.70211049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:49:45.027027035 +0000 UTC m=+71.960507756" watchObservedRunningTime="2026-09-07 03:49:45.028766979 +0000 UTC m=+71.962247699"
Sep 07 03:49:45.118734 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:45.118621    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:45.216375 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:45.216298    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:45.854581 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:45.854510    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:45.913197 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:45.913116    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:46.995052 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:46.995022    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5765d7cf8b-pjfrx"
Sep 07 03:49:47.262401 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:47.262313    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:47.314126 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:47.314089    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:47.417330 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:47.417294    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:47.516560 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:47.516474    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:51.007782 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:51.007729    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qbsbx" event={"ID":"4cf74d00-0e92-415e-8129-2033297d3707","Type":"ContainerStarted","Data":"6f29839ac5ff368b1214019dfa220b00ea493644c6862dc20a21277e333c7fd0"}
Sep 07 03:49:51.008283 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:51.007852    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:49:51.011891 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:51.011862    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"021172f4-4a0f-4980-8c47-1438394eec8d","Type":"ContainerStarted","Data":"bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72"}
Sep 07 03:49:51.012063 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:51.011938    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"021172f4-4a0f-4980-8c47-1438394eec8d","Type":"ContainerStarted","Data":"a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c"}
Sep 07 03:49:51.012063 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:51.011984    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"021172f4-4a0f-4980-8c47-1438394eec8d","Type":"ContainerStarted","Data":"cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4"}
Sep 07 03:49:51.012175 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:51.012077    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"021172f4-4a0f-4980-8c47-1438394eec8d","Type":"ContainerStarted","Data":"fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5"}
Sep 07 03:49:51.012175 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:51.012120    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"021172f4-4a0f-4980-8c47-1438394eec8d","Type":"ContainerStarted","Data":"2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70"}
Sep 07 03:49:51.012175 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:51.012151    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"021172f4-4a0f-4980-8c47-1438394eec8d","Type":"ContainerStarted","Data":"117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9"}
Sep 07 03:49:51.013588 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:51.013558    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-v9fzl" event={"ID":"244e7a50-68b4-41dc-aa4b-8b83813cf489","Type":"ContainerStarted","Data":"a1cd300bfb63b057e2fab3b28a7b031e74aa1774aa5823377ec197e7613a7cfe"}
Sep 07 03:49:51.024615 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:51.024565    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qbsbx" podStartSLOduration=68.374073105 podStartE2EDuration="1m17.024549232s" totalImagesPullingTime="8.650476127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:49:51.023346849 +0000 UTC m=+77.956827565" watchObservedRunningTime="2026-09-07 03:49:51.024549232 +0000 UTC m=+77.958029950"
Sep 07 03:49:51.038278 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:51.038226    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-v9fzl" podStartSLOduration=68.030392118 podStartE2EDuration="1m17.038192423s" totalImagesPullingTime="9.007800305s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 03:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:49:51.036865091 +0000 UTC m=+77.970345799" watchObservedRunningTime="2026-09-07 03:49:51.038192423 +0000 UTC m=+77.971673145"
Sep 07 03:49:51.612231 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:51.612185    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:51.672534 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:51.672500    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:52.049987 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:52.049930    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=7.953590093 podStartE2EDuration="19.049910462s" totalImagesPullingTime="11.052121454s" totalInitContainerRuntime="44.198915ms" isStatefulPod=false podCreationTimestamp="2026-09-07 03:49:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:49:52.048817471 +0000 UTC m=+78.982298190" watchObservedRunningTime="2026-09-07 03:49:52.049910462 +0000 UTC m=+78.983391217"
Sep 07 03:49:52.877333 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:52.877297    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:52.932328 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:52.932293    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:52.982452 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:52.982419    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:49:54.205681 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:49:54.205645    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:03.162735 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:03.162697    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:50:06.605376 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:06.605334    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:50:06.660098 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:06.660062    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:50:08.295659 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:08.295624    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:50:08.396296 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:08.396263    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:50:22.020335 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:22.020298    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qbsbx"
Sep 07 03:50:33.786072 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:33.786034    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:50:34.205796 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:34.205758    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:34.281843 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:34.281815    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:35.203371 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:35.203341    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:43.753591 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:43.753552    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:50:43.810359 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:43.810323    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:50:52.254550 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.254508    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 03:50:52.255041 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.254921    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="kube-rbac-proxy-web" containerID="cri-o://cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4" gracePeriod=600
Sep 07 03:50:52.255041 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.254990    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="config-reloader" containerID="cri-o://2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70" gracePeriod=600
Sep 07 03:50:52.255170 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.254911    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="kube-rbac-proxy-thanos" containerID="cri-o://bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72" gracePeriod=600
Sep 07 03:50:52.255170 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.255087    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="kube-rbac-proxy" containerID="cri-o://a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c" gracePeriod=600
Sep 07 03:50:52.255296 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.255171    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="thanos-sidecar" containerID="cri-o://fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5" gracePeriod=600
Sep 07 03:50:52.255296 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.254903    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="prometheus" containerID="cri-o://117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9" gracePeriod=600
Sep 07 03:50:52.510324 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.510260    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:52.666993 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.666952    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-thanos-prometheus-http-client-file\") pod \"021172f4-4a0f-4980-8c47-1438394eec8d\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") "
Sep 07 03:50:52.666993 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.666991    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-rulefiles-2\") pod \"021172f4-4a0f-4980-8c47-1438394eec8d\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") "
Sep 07 03:50:52.667205 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.667014    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-kube-rbac-proxy\") pod \"021172f4-4a0f-4980-8c47-1438394eec8d\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") "
Sep 07 03:50:52.667205 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.667045    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-grpc-tls\") pod \"021172f4-4a0f-4980-8c47-1438394eec8d\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") "
Sep 07 03:50:52.667205 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.667080    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/021172f4-4a0f-4980-8c47-1438394eec8d-config-out\") pod \"021172f4-4a0f-4980-8c47-1438394eec8d\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") "
Sep 07 03:50:52.667205 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.667112    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-db\") pod \"021172f4-4a0f-4980-8c47-1438394eec8d\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") "
Sep 07 03:50:52.667205 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.667133    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-config\") pod \"021172f4-4a0f-4980-8c47-1438394eec8d\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") "
Sep 07 03:50:52.667205 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.667164    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-prometheus-k8s-tls\") pod \"021172f4-4a0f-4980-8c47-1438394eec8d\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") "
Sep 07 03:50:52.667518 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.667286    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-configmap-metrics-client-ca\") pod \"021172f4-4a0f-4980-8c47-1438394eec8d\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") "
Sep 07 03:50:52.667518 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.667326    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-metrics-client-certs\") pod \"021172f4-4a0f-4980-8c47-1438394eec8d\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") "
Sep 07 03:50:52.667518 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.667350    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-configmap-kubelet-serving-ca-bundle\") pod \"021172f4-4a0f-4980-8c47-1438394eec8d\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") "
Sep 07 03:50:52.667518 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.667381    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"021172f4-4a0f-4980-8c47-1438394eec8d\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") "
Sep 07 03:50:52.667518 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.667375    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "021172f4-4a0f-4980-8c47-1438394eec8d" (UID: "021172f4-4a0f-4980-8c47-1438394eec8d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 03:50:52.667518 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.667414    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-configmap-serving-certs-ca-bundle\") pod \"021172f4-4a0f-4980-8c47-1438394eec8d\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") "
Sep 07 03:50:52.667518 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.667438    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bcwp\" (UniqueName: \"kubernetes.io/projected/021172f4-4a0f-4980-8c47-1438394eec8d-kube-api-access-2bcwp\") pod \"021172f4-4a0f-4980-8c47-1438394eec8d\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") "
Sep 07 03:50:52.667518 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.667485    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-rulefiles-1\") pod \"021172f4-4a0f-4980-8c47-1438394eec8d\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") "
Sep 07 03:50:52.667518 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.667507    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-trusted-ca-bundle\") pod \"021172f4-4a0f-4980-8c47-1438394eec8d\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") "
Sep 07 03:50:52.668122 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.667534    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"021172f4-4a0f-4980-8c47-1438394eec8d\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") "
Sep 07 03:50:52.668122 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.667566    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-web-config\") pod \"021172f4-4a0f-4980-8c47-1438394eec8d\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") "
Sep 07 03:50:52.668122 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.667593    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/021172f4-4a0f-4980-8c47-1438394eec8d-tls-assets\") pod \"021172f4-4a0f-4980-8c47-1438394eec8d\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") "
Sep 07 03:50:52.668122 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.667636    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-rulefiles-0\") pod \"021172f4-4a0f-4980-8c47-1438394eec8d\" (UID: \"021172f4-4a0f-4980-8c47-1438394eec8d\") "
Sep 07 03:50:52.668122 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.668025    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:50:52.668614 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.668580    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "021172f4-4a0f-4980-8c47-1438394eec8d" (UID: "021172f4-4a0f-4980-8c47-1438394eec8d"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 03:50:52.669770 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.668838    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "021172f4-4a0f-4980-8c47-1438394eec8d" (UID: "021172f4-4a0f-4980-8c47-1438394eec8d"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 03:50:52.669770 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.669025    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "021172f4-4a0f-4980-8c47-1438394eec8d" (UID: "021172f4-4a0f-4980-8c47-1438394eec8d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 03:50:52.669770 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.669106    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "021172f4-4a0f-4980-8c47-1438394eec8d" (UID: "021172f4-4a0f-4980-8c47-1438394eec8d"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 03:50:52.670131 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.670108    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "021172f4-4a0f-4980-8c47-1438394eec8d" (UID: "021172f4-4a0f-4980-8c47-1438394eec8d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 03:50:52.670262 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.670115    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "021172f4-4a0f-4980-8c47-1438394eec8d" (UID: "021172f4-4a0f-4980-8c47-1438394eec8d"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 03:50:52.670812 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.670420    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "021172f4-4a0f-4980-8c47-1438394eec8d" (UID: "021172f4-4a0f-4980-8c47-1438394eec8d"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 03:50:52.670927 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.670821    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "021172f4-4a0f-4980-8c47-1438394eec8d" (UID: "021172f4-4a0f-4980-8c47-1438394eec8d"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 03:50:52.671599 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.671576    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-config" (OuterVolumeSpecName: "config") pod "021172f4-4a0f-4980-8c47-1438394eec8d" (UID: "021172f4-4a0f-4980-8c47-1438394eec8d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 03:50:52.671862 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.671836    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/021172f4-4a0f-4980-8c47-1438394eec8d-config-out" (OuterVolumeSpecName: "config-out") pod "021172f4-4a0f-4980-8c47-1438394eec8d" (UID: "021172f4-4a0f-4980-8c47-1438394eec8d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 03:50:52.672085 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.672034    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "021172f4-4a0f-4980-8c47-1438394eec8d" (UID: "021172f4-4a0f-4980-8c47-1438394eec8d"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 03:50:52.672282 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.672246    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "021172f4-4a0f-4980-8c47-1438394eec8d" (UID: "021172f4-4a0f-4980-8c47-1438394eec8d"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 03:50:52.672572 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.672552    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "021172f4-4a0f-4980-8c47-1438394eec8d" (UID: "021172f4-4a0f-4980-8c47-1438394eec8d"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 03:50:52.672702 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.672678    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "021172f4-4a0f-4980-8c47-1438394eec8d" (UID: "021172f4-4a0f-4980-8c47-1438394eec8d"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 03:50:52.672978 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.672958    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "021172f4-4a0f-4980-8c47-1438394eec8d" (UID: "021172f4-4a0f-4980-8c47-1438394eec8d"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 03:50:52.673575 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.673550    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/021172f4-4a0f-4980-8c47-1438394eec8d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "021172f4-4a0f-4980-8c47-1438394eec8d" (UID: "021172f4-4a0f-4980-8c47-1438394eec8d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 03:50:52.673857 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.673839    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "021172f4-4a0f-4980-8c47-1438394eec8d" (UID: "021172f4-4a0f-4980-8c47-1438394eec8d"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 03:50:52.673910 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.673865    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/021172f4-4a0f-4980-8c47-1438394eec8d-kube-api-access-2bcwp" (OuterVolumeSpecName: "kube-api-access-2bcwp") pod "021172f4-4a0f-4980-8c47-1438394eec8d" (UID: "021172f4-4a0f-4980-8c47-1438394eec8d"). InnerVolumeSpecName "kube-api-access-2bcwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 03:50:52.683656 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.683629    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-web-config" (OuterVolumeSpecName: "web-config") pod "021172f4-4a0f-4980-8c47-1438394eec8d" (UID: "021172f4-4a0f-4980-8c47-1438394eec8d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 03:50:52.769233 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.769121    2583 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-metrics-client-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:50:52.769233 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.769159    2583 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:50:52.769233 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.769175    2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:50:52.769233 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.769187    2583 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:50:52.769233 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.769200    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bcwp\" (UniqueName: \"kubernetes.io/projected/021172f4-4a0f-4980-8c47-1438394eec8d-kube-api-access-2bcwp\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:50:52.769516 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.769238    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:50:52.769516 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.769252    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-trusted-ca-bundle\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:50:52.769516 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.769263    2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:50:52.769516 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.769275    2583 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-web-config\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:50:52.769516 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.769285    2583 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/021172f4-4a0f-4980-8c47-1438394eec8d-tls-assets\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:50:52.769516 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.769296    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:50:52.769516 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.769308    2583 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-thanos-prometheus-http-client-file\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:50:52.769516 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.769320    2583 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-kube-rbac-proxy\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:50:52.769516 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.769331    2583 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-grpc-tls\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:50:52.769516 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.769343    2583 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/021172f4-4a0f-4980-8c47-1438394eec8d-config-out\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:50:52.769516 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.769354    2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/021172f4-4a0f-4980-8c47-1438394eec8d-prometheus-k8s-db\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:50:52.769516 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.769366    2583 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-config\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:50:52.769516 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.769379    2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/021172f4-4a0f-4980-8c47-1438394eec8d-secret-prometheus-k8s-tls\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:50:52.769516 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:52.769392    2583 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/021172f4-4a0f-4980-8c47-1438394eec8d-configmap-metrics-client-ca\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:50:53.261149 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.261111    2583 generic.go:356] "Generic (PLEG): container finished" podID="021172f4-4a0f-4980-8c47-1438394eec8d" containerID="bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72" exitCode=0
Sep 07 03:50:53.261149 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.261140    2583 generic.go:356] "Generic (PLEG): container finished" podID="021172f4-4a0f-4980-8c47-1438394eec8d" containerID="a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c" exitCode=0
Sep 07 03:50:53.261149 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.261146    2583 generic.go:356] "Generic (PLEG): container finished" podID="021172f4-4a0f-4980-8c47-1438394eec8d" containerID="cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4" exitCode=0
Sep 07 03:50:53.261149 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.261152    2583 generic.go:356] "Generic (PLEG): container finished" podID="021172f4-4a0f-4980-8c47-1438394eec8d" containerID="fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5" exitCode=0
Sep 07 03:50:53.261149 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.261157    2583 generic.go:356] "Generic (PLEG): container finished" podID="021172f4-4a0f-4980-8c47-1438394eec8d" containerID="2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70" exitCode=0
Sep 07 03:50:53.261149 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.261163    2583 generic.go:356] "Generic (PLEG): container finished" podID="021172f4-4a0f-4980-8c47-1438394eec8d" containerID="117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9" exitCode=0
Sep 07 03:50:53.261879 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.261204    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.261879 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.261201    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"021172f4-4a0f-4980-8c47-1438394eec8d","Type":"ContainerDied","Data":"bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72"}
Sep 07 03:50:53.261879 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.261362    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"021172f4-4a0f-4980-8c47-1438394eec8d","Type":"ContainerDied","Data":"a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c"}
Sep 07 03:50:53.261879 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.261390    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"021172f4-4a0f-4980-8c47-1438394eec8d","Type":"ContainerDied","Data":"cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4"}
Sep 07 03:50:53.261879 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.261416    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"021172f4-4a0f-4980-8c47-1438394eec8d","Type":"ContainerDied","Data":"fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5"}
Sep 07 03:50:53.261879 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.261435    2583 scope.go:117] "RemoveContainer" containerID="bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72"
Sep 07 03:50:53.261879 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.261444    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"021172f4-4a0f-4980-8c47-1438394eec8d","Type":"ContainerDied","Data":"2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70"}
Sep 07 03:50:53.261879 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.261470    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"021172f4-4a0f-4980-8c47-1438394eec8d","Type":"ContainerDied","Data":"117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9"}
Sep 07 03:50:53.261879 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.261493    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"021172f4-4a0f-4980-8c47-1438394eec8d","Type":"ContainerDied","Data":"5105a3501610a0cdc8af38d1483bf70fab1b46327163194f64aa3817f523a379"}
Sep 07 03:50:53.281476 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.281459    2583 scope.go:117] "RemoveContainer" containerID="a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c"
Sep 07 03:50:53.289502 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.289484    2583 scope.go:117] "RemoveContainer" containerID="cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4"
Sep 07 03:50:53.296189 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.296158    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 03:50:53.297552 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.297533    2583 scope.go:117] "RemoveContainer" containerID="fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5"
Sep 07 03:50:53.300391 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.300369    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 03:50:53.305049 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.305032    2583 scope.go:117] "RemoveContainer" containerID="2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70"
Sep 07 03:50:53.311900 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.311880    2583 scope.go:117] "RemoveContainer" containerID="117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9"
Sep 07 03:50:53.319865 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.319846    2583 scope.go:117] "RemoveContainer" containerID="6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802"
Sep 07 03:50:53.326354 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.326334    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 03:50:53.326599 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.326588    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="kube-rbac-proxy-thanos"
Sep 07 03:50:53.326639 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.326601    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="kube-rbac-proxy-thanos"
Sep 07 03:50:53.326639 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.326615    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="config-reloader"
Sep 07 03:50:53.326639 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.326620    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="config-reloader"
Sep 07 03:50:53.326639 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.326626    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="kube-rbac-proxy-web"
Sep 07 03:50:53.326639 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.326632    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="kube-rbac-proxy-web"
Sep 07 03:50:53.326639 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.326640    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="thanos-sidecar"
Sep 07 03:50:53.326805 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.326646    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="thanos-sidecar"
Sep 07 03:50:53.326805 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.326652    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="init-config-reloader"
Sep 07 03:50:53.326805 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.326657    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="init-config-reloader"
Sep 07 03:50:53.326805 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.326666    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="kube-rbac-proxy"
Sep 07 03:50:53.326805 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.326671    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="kube-rbac-proxy"
Sep 07 03:50:53.326805 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.326677    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="prometheus"
Sep 07 03:50:53.326805 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.326681    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="prometheus"
Sep 07 03:50:53.326805 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.326727    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="thanos-sidecar"
Sep 07 03:50:53.326805 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.326739    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="prometheus"
Sep 07 03:50:53.326805 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.326745    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="config-reloader"
Sep 07 03:50:53.326805 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.326750    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="kube-rbac-proxy-web"
Sep 07 03:50:53.326805 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.326756    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="kube-rbac-proxy"
Sep 07 03:50:53.326805 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.326761    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" containerName="kube-rbac-proxy-thanos"
Sep 07 03:50:53.327427 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.327409    2583 scope.go:117] "RemoveContainer" containerID="bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72"
Sep 07 03:50:53.327759 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:50:53.327739    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72\": container with ID starting with bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72 not found: ID does not exist" containerID="bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72"
Sep 07 03:50:53.327837 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.327766    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72"} err="failed to get container status \"bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72\": rpc error: code = NotFound desc = could not find container \"bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72\": container with ID starting with bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72 not found: ID does not exist"
Sep 07 03:50:53.327837 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.327798    2583 scope.go:117] "RemoveContainer" containerID="a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c"
Sep 07 03:50:53.328033 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:50:53.328018    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c\": container with ID starting with a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c not found: ID does not exist" containerID="a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c"
Sep 07 03:50:53.328069 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.328037    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c"} err="failed to get container status \"a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c\": rpc error: code = NotFound desc = could not find container \"a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c\": container with ID starting with a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c not found: ID does not exist"
Sep 07 03:50:53.328069 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.328051    2583 scope.go:117] "RemoveContainer" containerID="cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4"
Sep 07 03:50:53.328371 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:50:53.328351    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4\": container with ID starting with cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4 not found: ID does not exist" containerID="cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4"
Sep 07 03:50:53.328451 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.328376    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4"} err="failed to get container status \"cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4\": rpc error: code = NotFound desc = could not find container \"cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4\": container with ID starting with cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4 not found: ID does not exist"
Sep 07 03:50:53.328451 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.328395    2583 scope.go:117] "RemoveContainer" containerID="fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5"
Sep 07 03:50:53.328630 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:50:53.328611    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5\": container with ID starting with fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5 not found: ID does not exist" containerID="fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5"
Sep 07 03:50:53.328706 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.328630    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5"} err="failed to get container status \"fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5\": rpc error: code = NotFound desc = could not find container \"fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5\": container with ID starting with fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5 not found: ID does not exist"
Sep 07 03:50:53.328706 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.328642    2583 scope.go:117] "RemoveContainer" containerID="2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70"
Sep 07 03:50:53.328911 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:50:53.328897    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70\": container with ID starting with 2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70 not found: ID does not exist" containerID="2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70"
Sep 07 03:50:53.328951 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.328913    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70"} err="failed to get container status \"2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70\": rpc error: code = NotFound desc = could not find container \"2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70\": container with ID starting with 2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70 not found: ID does not exist"
Sep 07 03:50:53.328951 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.328924    2583 scope.go:117] "RemoveContainer" containerID="117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9"
Sep 07 03:50:53.329136 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:50:53.329120    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9\": container with ID starting with 117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9 not found: ID does not exist" containerID="117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9"
Sep 07 03:50:53.329181 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.329140    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9"} err="failed to get container status \"117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9\": rpc error: code = NotFound desc = could not find container \"117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9\": container with ID starting with 117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9 not found: ID does not exist"
Sep 07 03:50:53.329181 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.329152    2583 scope.go:117] "RemoveContainer" containerID="6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802"
Sep 07 03:50:53.329404 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:50:53.329388    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802\": container with ID starting with 6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802 not found: ID does not exist" containerID="6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802"
Sep 07 03:50:53.329445 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.329405    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802"} err="failed to get container status \"6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802\": rpc error: code = NotFound desc = could not find container \"6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802\": container with ID starting with 6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802 not found: ID does not exist"
Sep 07 03:50:53.329445 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.329416    2583 scope.go:117] "RemoveContainer" containerID="bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72"
Sep 07 03:50:53.329638 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.329622    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72"} err="failed to get container status \"bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72\": rpc error: code = NotFound desc = could not find container \"bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72\": container with ID starting with bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72 not found: ID does not exist"
Sep 07 03:50:53.329677 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.329639    2583 scope.go:117] "RemoveContainer" containerID="a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c"
Sep 07 03:50:53.329861 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.329842    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c"} err="failed to get container status \"a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c\": rpc error: code = NotFound desc = could not find container \"a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c\": container with ID starting with a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c not found: ID does not exist"
Sep 07 03:50:53.329913 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.329863    2583 scope.go:117] "RemoveContainer" containerID="cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4"
Sep 07 03:50:53.330102 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.330086    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4"} err="failed to get container status \"cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4\": rpc error: code = NotFound desc = could not find container \"cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4\": container with ID starting with cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4 not found: ID does not exist"
Sep 07 03:50:53.330146 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.330106    2583 scope.go:117] "RemoveContainer" containerID="fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5"
Sep 07 03:50:53.330146 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.330091    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.330447 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.330423    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5"} err="failed to get container status \"fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5\": rpc error: code = NotFound desc = could not find container \"fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5\": container with ID starting with fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5 not found: ID does not exist"
Sep 07 03:50:53.330447 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.330447    2583 scope.go:117] "RemoveContainer" containerID="2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70"
Sep 07 03:50:53.330806 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.330787    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70"} err="failed to get container status \"2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70\": rpc error: code = NotFound desc = could not find container \"2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70\": container with ID starting with 2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70 not found: ID does not exist"
Sep 07 03:50:53.330896 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.330808    2583 scope.go:117] "RemoveContainer" containerID="117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9"
Sep 07 03:50:53.331110 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.331089    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9"} err="failed to get container status \"117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9\": rpc error: code = NotFound desc = could not find container \"117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9\": container with ID starting with 117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9 not found: ID does not exist"
Sep 07 03:50:53.331165 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.331112    2583 scope.go:117] "RemoveContainer" containerID="6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802"
Sep 07 03:50:53.331347 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.331331    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802"} err="failed to get container status \"6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802\": rpc error: code = NotFound desc = could not find container \"6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802\": container with ID starting with 6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802 not found: ID does not exist"
Sep 07 03:50:53.331401 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.331348    2583 scope.go:117] "RemoveContainer" containerID="bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72"
Sep 07 03:50:53.331645 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.331565    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72"} err="failed to get container status \"bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72\": rpc error: code = NotFound desc = could not find container \"bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72\": container with ID starting with bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72 not found: ID does not exist"
Sep 07 03:50:53.331645 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.331636    2583 scope.go:117] "RemoveContainer" containerID="a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c"
Sep 07 03:50:53.331905 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.331888    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c"} err="failed to get container status \"a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c\": rpc error: code = NotFound desc = could not find container \"a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c\": container with ID starting with a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c not found: ID does not exist"
Sep 07 03:50:53.331949 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.331906    2583 scope.go:117] "RemoveContainer" containerID="cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4"
Sep 07 03:50:53.332153 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.332133    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4"} err="failed to get container status \"cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4\": rpc error: code = NotFound desc = could not find container \"cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4\": container with ID starting with cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4 not found: ID does not exist"
Sep 07 03:50:53.332153 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.332152    2583 scope.go:117] "RemoveContainer" containerID="fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5"
Sep 07 03:50:53.332409 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.332391    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5"} err="failed to get container status \"fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5\": rpc error: code = NotFound desc = could not find container \"fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5\": container with ID starting with fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5 not found: ID does not exist"
Sep 07 03:50:53.332468 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.332410    2583 scope.go:117] "RemoveContainer" containerID="2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70"
Sep 07 03:50:53.332722 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.332705    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 07 03:50:53.332810 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.332765    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 07 03:50:53.332867 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.332797    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70"} err="failed to get container status \"2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70\": rpc error: code = NotFound desc = could not find container \"2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70\": container with ID starting with 2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70 not found: ID does not exist"
Sep 07 03:50:53.332867 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.332822    2583 scope.go:117] "RemoveContainer" containerID="117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9"
Sep 07 03:50:53.332968 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.332880    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 07 03:50:53.333158 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.333135    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9"} err="failed to get container status \"117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9\": rpc error: code = NotFound desc = could not find container \"117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9\": container with ID starting with 117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9 not found: ID does not exist"
Sep 07 03:50:53.333383 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.333366    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-72t2596m39tvm\""
Sep 07 03:50:53.333447 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.333366    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 07 03:50:53.333521 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.333378    2583 scope.go:117] "RemoveContainer" containerID="6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802"
Sep 07 03:50:53.333646 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.333620    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 07 03:50:53.333801 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.333777    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 07 03:50:53.333801 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.333796    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 07 03:50:53.333956 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.333785    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 07 03:50:53.334015 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.333983    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802"} err="failed to get container status \"6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802\": rpc error: code = NotFound desc = could not find container \"6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802\": container with ID starting with 6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802 not found: ID does not exist"
Sep 07 03:50:53.334015 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.334005    2583 scope.go:117] "RemoveContainer" containerID="bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72"
Sep 07 03:50:53.334372 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.334334    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72"} err="failed to get container status \"bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72\": rpc error: code = NotFound desc = could not find container \"bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72\": container with ID starting with bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72 not found: ID does not exist"
Sep 07 03:50:53.334372 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.334360    2583 scope.go:117] "RemoveContainer" containerID="a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c"
Sep 07 03:50:53.334523 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.334446    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 07 03:50:53.334523 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.334465    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 07 03:50:53.334523 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.334496    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-w62b8\""
Sep 07 03:50:53.334668 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.334613    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 07 03:50:53.334952 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.334801    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c"} err="failed to get container status \"a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c\": rpc error: code = NotFound desc = could not find container \"a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c\": container with ID starting with a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c not found: ID does not exist"
Sep 07 03:50:53.334952 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.334826    2583 scope.go:117] "RemoveContainer" containerID="cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4"
Sep 07 03:50:53.334952 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.334857    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 07 03:50:53.335135 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.335120    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 07 03:50:53.335955 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.335929    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4"} err="failed to get container status \"cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4\": rpc error: code = NotFound desc = could not find container \"cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4\": container with ID starting with cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4 not found: ID does not exist"
Sep 07 03:50:53.336060 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.335958    2583 scope.go:117] "RemoveContainer" containerID="fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5"
Sep 07 03:50:53.336256 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.336236    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5"} err="failed to get container status \"fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5\": rpc error: code = NotFound desc = could not find container \"fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5\": container with ID starting with fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5 not found: ID does not exist"
Sep 07 03:50:53.336311 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.336257    2583 scope.go:117] "RemoveContainer" containerID="2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70"
Sep 07 03:50:53.336510 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.336457    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70"} err="failed to get container status \"2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70\": rpc error: code = NotFound desc = could not find container \"2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70\": container with ID starting with 2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70 not found: ID does not exist"
Sep 07 03:50:53.336510 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.336482    2583 scope.go:117] "RemoveContainer" containerID="117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9"
Sep 07 03:50:53.336744 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.336721    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9"} err="failed to get container status \"117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9\": rpc error: code = NotFound desc = could not find container \"117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9\": container with ID starting with 117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9 not found: ID does not exist"
Sep 07 03:50:53.336783 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.336745    2583 scope.go:117] "RemoveContainer" containerID="6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802"
Sep 07 03:50:53.336827 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.336818    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 07 03:50:53.337607 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.337535    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802"} err="failed to get container status \"6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802\": rpc error: code = NotFound desc = could not find container \"6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802\": container with ID starting with 6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802 not found: ID does not exist"
Sep 07 03:50:53.337607 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.337569    2583 scope.go:117] "RemoveContainer" containerID="bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72"
Sep 07 03:50:53.338460 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.338426    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72"} err="failed to get container status \"bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72\": rpc error: code = NotFound desc = could not find container \"bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72\": container with ID starting with bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72 not found: ID does not exist"
Sep 07 03:50:53.338460 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.338458    2583 scope.go:117] "RemoveContainer" containerID="a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c"
Sep 07 03:50:53.338955 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.338930    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c"} err="failed to get container status \"a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c\": rpc error: code = NotFound desc = could not find container \"a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c\": container with ID starting with a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c not found: ID does not exist"
Sep 07 03:50:53.339036 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.338959    2583 scope.go:117] "RemoveContainer" containerID="cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4"
Sep 07 03:50:53.339388 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.339343    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4"} err="failed to get container status \"cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4\": rpc error: code = NotFound desc = could not find container \"cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4\": container with ID starting with cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4 not found: ID does not exist"
Sep 07 03:50:53.339388 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.339374    2583 scope.go:117] "RemoveContainer" containerID="fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5"
Sep 07 03:50:53.339735 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.339709    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5"} err="failed to get container status \"fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5\": rpc error: code = NotFound desc = could not find container \"fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5\": container with ID starting with fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5 not found: ID does not exist"
Sep 07 03:50:53.339817 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.339746    2583 scope.go:117] "RemoveContainer" containerID="2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70"
Sep 07 03:50:53.340109 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.340083    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70"} err="failed to get container status \"2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70\": rpc error: code = NotFound desc = could not find container \"2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70\": container with ID starting with 2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70 not found: ID does not exist"
Sep 07 03:50:53.340188 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.340113    2583 scope.go:117] "RemoveContainer" containerID="117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9"
Sep 07 03:50:53.342311 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.342291    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 07 03:50:53.343898 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.343871    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9"} err="failed to get container status \"117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9\": rpc error: code = NotFound desc = could not find container \"117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9\": container with ID starting with 117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9 not found: ID does not exist"
Sep 07 03:50:53.343994 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.343900    2583 scope.go:117] "RemoveContainer" containerID="6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802"
Sep 07 03:50:53.344297 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.344234    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802"} err="failed to get container status \"6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802\": rpc error: code = NotFound desc = could not find container \"6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802\": container with ID starting with 6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802 not found: ID does not exist"
Sep 07 03:50:53.344297 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.344266    2583 scope.go:117] "RemoveContainer" containerID="bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72"
Sep 07 03:50:53.344839 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.344811    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72"} err="failed to get container status \"bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72\": rpc error: code = NotFound desc = could not find container \"bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72\": container with ID starting with bbbc13333e3bd9052c33e9349e27e4647766d41e4d0228df6455cfe29ed22f72 not found: ID does not exist"
Sep 07 03:50:53.344988 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.344842    2583 scope.go:117] "RemoveContainer" containerID="a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c"
Sep 07 03:50:53.345636 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.345601    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c"} err="failed to get container status \"a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c\": rpc error: code = NotFound desc = could not find container \"a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c\": container with ID starting with a1bf6bf07badb5f675e7402172a95b618dd2d2b0fe2593eb887e9bba788c1a9c not found: ID does not exist"
Sep 07 03:50:53.345730 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.345641    2583 scope.go:117] "RemoveContainer" containerID="cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4"
Sep 07 03:50:53.346001 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.345976    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4"} err="failed to get container status \"cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4\": rpc error: code = NotFound desc = could not find container \"cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4\": container with ID starting with cea43a4cb115c057b6dc62e20107f539def5b4ef30683ef7349586f738c2dff4 not found: ID does not exist"
Sep 07 03:50:53.346001 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.346003    2583 scope.go:117] "RemoveContainer" containerID="fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5"
Sep 07 03:50:53.346317 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.346288    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5"} err="failed to get container status \"fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5\": rpc error: code = NotFound desc = could not find container \"fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5\": container with ID starting with fe167c157f66ca803c7c284b5a0fec8b3c993506f9892040b37839abe3f883e5 not found: ID does not exist"
Sep 07 03:50:53.346413 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.346315    2583 scope.go:117] "RemoveContainer" containerID="2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70"
Sep 07 03:50:53.346627 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.346601    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70"} err="failed to get container status \"2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70\": rpc error: code = NotFound desc = could not find container \"2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70\": container with ID starting with 2f5c704821b39f9956dcb4b6700eb939cbab76d4cc9e32887f7da9fd41e60d70 not found: ID does not exist"
Sep 07 03:50:53.346725 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.346636    2583 scope.go:117] "RemoveContainer" containerID="117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9"
Sep 07 03:50:53.346906 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.346885    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9"} err="failed to get container status \"117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9\": rpc error: code = NotFound desc = could not find container \"117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9\": container with ID starting with 117f42eae63eaf365f2dbf2f12c5a386bb9341280fd7cfe08e9fb876ffcc15f9 not found: ID does not exist"
Sep 07 03:50:53.346906 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.346906    2583 scope.go:117] "RemoveContainer" containerID="6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802"
Sep 07 03:50:53.347178 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.347161    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802"} err="failed to get container status \"6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802\": rpc error: code = NotFound desc = could not find container \"6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802\": container with ID starting with 6fd6718a4a76dca066318c36593f72211e1ae151189fe7750f629d24f64f1802 not found: ID does not exist"
Sep 07 03:50:53.347333 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.347318    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 03:50:53.474968 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.474928    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6718f4c7-fcaa-4398-b866-dd03be6f1a57-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.474968 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.474972    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6718f4c7-fcaa-4398-b866-dd03be6f1a57-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.475182 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.474994    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-config\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.475182 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.475011    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6718f4c7-fcaa-4398-b866-dd03be6f1a57-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.475182 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.475078    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6718f4c7-fcaa-4398-b866-dd03be6f1a57-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.475182 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.475113    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.475182 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.475132    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6718f4c7-fcaa-4398-b866-dd03be6f1a57-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.475182 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.475151    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6718f4c7-fcaa-4398-b866-dd03be6f1a57-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.475182 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.475172    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6718f4c7-fcaa-4398-b866-dd03be6f1a57-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.475410 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.475251    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6shf\" (UniqueName: \"kubernetes.io/projected/6718f4c7-fcaa-4398-b866-dd03be6f1a57-kube-api-access-z6shf\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.475410 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.475268    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6718f4c7-fcaa-4398-b866-dd03be6f1a57-config-out\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.475410 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.475290    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.475410 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.475329    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.475410 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.475382    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.475552 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.475408    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-web-config\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.475552 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.475456    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.475552 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.475471    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.475552 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.475484    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6718f4c7-fcaa-4398-b866-dd03be6f1a57-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.475552 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.475502    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6718f4c7-fcaa-4398-b866-dd03be6f1a57-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.475552 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.475538    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.521090 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.521008    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="021172f4-4a0f-4980-8c47-1438394eec8d" path="/var/lib/kubelet/pods/021172f4-4a0f-4980-8c47-1438394eec8d/volumes"
Sep 07 03:50:53.576117 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.576078    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.576293 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.576119    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.576293 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.576138    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-web-config\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.576293 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.576157    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.576293 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.576176    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.576293 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.576200    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6718f4c7-fcaa-4398-b866-dd03be6f1a57-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.576293 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.576250    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6718f4c7-fcaa-4398-b866-dd03be6f1a57-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.576293 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.576280    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.576562 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.576309    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6718f4c7-fcaa-4398-b866-dd03be6f1a57-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.576562 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.576331    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6718f4c7-fcaa-4398-b866-dd03be6f1a57-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.576562 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.576357    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-config\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.576562 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.576371    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6718f4c7-fcaa-4398-b866-dd03be6f1a57-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.576562 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.576391    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6718f4c7-fcaa-4398-b866-dd03be6f1a57-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.576562 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.576405    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.576562 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.576428    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6718f4c7-fcaa-4398-b866-dd03be6f1a57-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.576562 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.576452    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6718f4c7-fcaa-4398-b866-dd03be6f1a57-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.576562 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.576473    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6718f4c7-fcaa-4398-b866-dd03be6f1a57-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.576562 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.576504    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6shf\" (UniqueName: \"kubernetes.io/projected/6718f4c7-fcaa-4398-b866-dd03be6f1a57-kube-api-access-z6shf\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.576562 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.576524    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6718f4c7-fcaa-4398-b866-dd03be6f1a57-config-out\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.576562 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.576543    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.577313 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.576917    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6718f4c7-fcaa-4398-b866-dd03be6f1a57-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.579647 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.577874    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6718f4c7-fcaa-4398-b866-dd03be6f1a57-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.579647 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.577890    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6718f4c7-fcaa-4398-b866-dd03be6f1a57-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.579647 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.578511    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6718f4c7-fcaa-4398-b866-dd03be6f1a57-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.579889 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.579722    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6718f4c7-fcaa-4398-b866-dd03be6f1a57-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.580580 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.580174    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.580580 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.580421    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6718f4c7-fcaa-4398-b866-dd03be6f1a57-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.580580 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.580486    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6718f4c7-fcaa-4398-b866-dd03be6f1a57-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.580808 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.580746    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.580867 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.580852    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.581055 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.581012    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.581453 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.581431    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.581547 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.581465    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.581659 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.581632    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6718f4c7-fcaa-4398-b866-dd03be6f1a57-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.582165 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.582140    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6718f4c7-fcaa-4398-b866-dd03be6f1a57-config-out\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.582270 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.582243    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.582807 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.582782    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6718f4c7-fcaa-4398-b866-dd03be6f1a57-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.582949 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.582932    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-config\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.582949 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.582943    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6718f4c7-fcaa-4398-b866-dd03be6f1a57-web-config\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.587179 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.587159    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6shf\" (UniqueName: \"kubernetes.io/projected/6718f4c7-fcaa-4398-b866-dd03be6f1a57-kube-api-access-z6shf\") pod \"prometheus-k8s-0\" (UID: \"6718f4c7-fcaa-4398-b866-dd03be6f1a57\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.647926 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.647879    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:50:53.930197 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:53.930165    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 07 03:50:53.932570 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:50:53.932545    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6718f4c7_fcaa_4398_b866_dd03be6f1a57.slice/crio-a6d00364a9d439a2a264e2cab2b66e663ea24d4a8f43fb36774b6c8172f39cd4 WatchSource:0}: Error finding container a6d00364a9d439a2a264e2cab2b66e663ea24d4a8f43fb36774b6c8172f39cd4: Status 404 returned error can't find the container with id a6d00364a9d439a2a264e2cab2b66e663ea24d4a8f43fb36774b6c8172f39cd4
Sep 07 03:50:54.267134 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:54.267098    2583 generic.go:356] "Generic (PLEG): container finished" podID="6718f4c7-fcaa-4398-b866-dd03be6f1a57" containerID="513e8b00b11f12a88679d0f35fc9e3b839aa543ed736ab237b2bbeae1cad30d2" exitCode=0
Sep 07 03:50:54.267498 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:54.267185    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6718f4c7-fcaa-4398-b866-dd03be6f1a57","Type":"ContainerDied","Data":"513e8b00b11f12a88679d0f35fc9e3b839aa543ed736ab237b2bbeae1cad30d2"}
Sep 07 03:50:54.267498 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:54.267277    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6718f4c7-fcaa-4398-b866-dd03be6f1a57","Type":"ContainerStarted","Data":"a6d00364a9d439a2a264e2cab2b66e663ea24d4a8f43fb36774b6c8172f39cd4"}
Sep 07 03:50:55.278538 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:55.278504    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6718f4c7-fcaa-4398-b866-dd03be6f1a57","Type":"ContainerStarted","Data":"4df5b384c3969fbb2f9111c91d709b64a43c83ae4d1eab8c1057518aa22c6a5c"}
Sep 07 03:50:55.278923 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:55.278599    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6718f4c7-fcaa-4398-b866-dd03be6f1a57","Type":"ContainerStarted","Data":"832a520678db918f0137dce33d13626982f6fa3b7036822907b0f1202dd1fec6"}
Sep 07 03:50:55.278923 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:55.278625    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6718f4c7-fcaa-4398-b866-dd03be6f1a57","Type":"ContainerStarted","Data":"1c355118176e08f84ee9a4f92ac32e411fe5008f4ee11be8f319763263eac2b0"}
Sep 07 03:50:55.278923 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:55.278649    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6718f4c7-fcaa-4398-b866-dd03be6f1a57","Type":"ContainerStarted","Data":"6134514b4ac5d419c06f7b5ca8b085505fcc26f1f3d1be414eeee55b5e2fbb72"}
Sep 07 03:50:55.278923 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:55.278672    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6718f4c7-fcaa-4398-b866-dd03be6f1a57","Type":"ContainerStarted","Data":"92b5fe12f89ed100d7fb5fa6e9ec304c3d139d05fd81d301939bcfdeff977c43"}
Sep 07 03:50:55.278923 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:55.278695    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6718f4c7-fcaa-4398-b866-dd03be6f1a57","Type":"ContainerStarted","Data":"21985ab8cafcb1e7b4adf6f6bbfc9c44602606f044d27124ce39475159fb9f2d"}
Sep 07 03:50:55.307862 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:55.307804    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.289798409 podStartE2EDuration="2.307786014s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.987605ms" isStatefulPod=false podCreationTimestamp="2026-09-07 03:50:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:50:55.307077215 +0000 UTC m=+142.240557935" watchObservedRunningTime="2026-09-07 03:50:55.307786014 +0000 UTC m=+142.241266732"
Sep 07 03:50:55.606167 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:55.606083    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:50:55.655052 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:55.655012    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:50:58.648025 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:50:58.647968    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:51:02.986972 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:51:02.986935    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:51:03.088705 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:51:03.088670    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:51:10.045968 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:51:10.045927    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:51:10.099101 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:51:10.099067    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:51:10.195873 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:51:10.195841    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:51:10.250458 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:51:10.250419    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:51:10.990484 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:51:10.990449    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:51:11.092643 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:51:11.092612    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:51:21.218110 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:51:21.218076    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:51:21.276509 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:51:21.276474    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:51:21.331963 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:51:21.331926    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:51:22.266310 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:51:22.266273    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:51:53.649063 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:51:53.649011    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:51:53.665107 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:51:53.665082    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:51:54.341810 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:51:54.341775    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:51:54.438750 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:51:54.438717    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:51:54.526300 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:51:54.526276    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 07 03:52:16.064265 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:16.064233    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:16.913312 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:16.913274    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:19.674046 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:19.674008    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:19.776654 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:19.776618    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:21.552686 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:21.552654    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:21.666348 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:21.666299    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:21.770272 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:21.770240    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:23.948865 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:23.948834    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:24.347476 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:24.347387    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:26.470434 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:26.470397    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:26.533833 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:26.533795    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:31.149388 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:31.149348    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:31.914269 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:31.914233    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-h7z7c"]
Sep 07 03:52:31.916457 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:31.916437    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-h7z7c"
Sep 07 03:52:31.918926 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:31.918907    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-5jn48\""
Sep 07 03:52:31.919197 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:31.919178    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 07 03:52:31.919982 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:31.919965    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 07 03:52:31.926488 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:31.926467    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-h7z7c"]
Sep 07 03:52:31.964052 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:31.964010    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/519761b7-6227-495a-a53b-181421430eaa-bound-sa-token\") pod \"cert-manager-55b4d954d5-h7z7c\" (UID: \"519761b7-6227-495a-a53b-181421430eaa\") " pod="cert-manager/cert-manager-55b4d954d5-h7z7c"
Sep 07 03:52:31.964263 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:31.964068    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqspc\" (UniqueName: \"kubernetes.io/projected/519761b7-6227-495a-a53b-181421430eaa-kube-api-access-wqspc\") pod \"cert-manager-55b4d954d5-h7z7c\" (UID: \"519761b7-6227-495a-a53b-181421430eaa\") " pod="cert-manager/cert-manager-55b4d954d5-h7z7c"
Sep 07 03:52:32.064695 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:32.064657    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/519761b7-6227-495a-a53b-181421430eaa-bound-sa-token\") pod \"cert-manager-55b4d954d5-h7z7c\" (UID: \"519761b7-6227-495a-a53b-181421430eaa\") " pod="cert-manager/cert-manager-55b4d954d5-h7z7c"
Sep 07 03:52:32.064906 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:32.064722    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqspc\" (UniqueName: \"kubernetes.io/projected/519761b7-6227-495a-a53b-181421430eaa-kube-api-access-wqspc\") pod \"cert-manager-55b4d954d5-h7z7c\" (UID: \"519761b7-6227-495a-a53b-181421430eaa\") " pod="cert-manager/cert-manager-55b4d954d5-h7z7c"
Sep 07 03:52:32.073159 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:32.073123    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/519761b7-6227-495a-a53b-181421430eaa-bound-sa-token\") pod \"cert-manager-55b4d954d5-h7z7c\" (UID: \"519761b7-6227-495a-a53b-181421430eaa\") " pod="cert-manager/cert-manager-55b4d954d5-h7z7c"
Sep 07 03:52:32.073582 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:32.073561    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqspc\" (UniqueName: \"kubernetes.io/projected/519761b7-6227-495a-a53b-181421430eaa-kube-api-access-wqspc\") pod \"cert-manager-55b4d954d5-h7z7c\" (UID: \"519761b7-6227-495a-a53b-181421430eaa\") " pod="cert-manager/cert-manager-55b4d954d5-h7z7c"
Sep 07 03:52:32.236010 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:32.235974    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-h7z7c"
Sep 07 03:52:32.506717 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:32.506641    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-h7z7c"]
Sep 07 03:52:32.509905 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:52:32.509882    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod519761b7_6227_495a_a53b_181421430eaa.slice/crio-ec95c07feb301efd9f2201a0c1a67584d62d45022bb987ff5305b8aa66b6ecae WatchSource:0}: Error finding container ec95c07feb301efd9f2201a0c1a67584d62d45022bb987ff5305b8aa66b6ecae: Status 404 returned error can't find the container with id ec95c07feb301efd9f2201a0c1a67584d62d45022bb987ff5305b8aa66b6ecae
Sep 07 03:52:32.655313 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:32.655273    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-h7z7c" event={"ID":"519761b7-6227-495a-a53b-181421430eaa","Type":"ContainerStarted","Data":"ec95c07feb301efd9f2201a0c1a67584d62d45022bb987ff5305b8aa66b6ecae"}
Sep 07 03:52:34.147453 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:34.147416    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:34.248301 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:34.248262    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:35.668887 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:35.668793    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-h7z7c" event={"ID":"519761b7-6227-495a-a53b-181421430eaa","Type":"ContainerStarted","Data":"7e448e0f49cbeea2bb5cb5b7372a031ded1c28f4f8e4e8028fffdac7c46dcc51"}
Sep 07 03:52:35.684953 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:35.684899    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-h7z7c" podStartSLOduration=1.96651459 podStartE2EDuration="4.684883871s" totalImagesPullingTime="2.718369281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:52:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:52:35.684696645 +0000 UTC m=+242.618177363" watchObservedRunningTime="2026-09-07 03:52:35.684883871 +0000 UTC m=+242.618364590"
Sep 07 03:52:40.044426 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:40.044392    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:40.145523 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:40.145491    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:42.044367 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:42.044332    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:48.974038 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:48.974003    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:49.026626 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:49.026593    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:55.724029 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:55.723956    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:55.822542 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:55.822514    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:56.318719 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.318690    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:56.559829 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.559796    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:56.634463 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.634385    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:56.655183 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.655151    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd"]
Sep 07 03:52:56.658627 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.658607    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd"
Sep 07 03:52:56.662833 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.662814    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 07 03:52:56.666290 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.666269    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 07 03:52:56.666395 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.666297    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7q6jm\""
Sep 07 03:52:56.666395 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.666274    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 07 03:52:56.666395 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.666351    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 07 03:52:56.691430 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.691399    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd"]
Sep 07 03:52:56.769934 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.769892    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/863f75b6-d5da-405e-a695-be1c0eb08446-apiservice-cert\") pod \"opendatahub-operator-controller-manager-756ff4ddd-m2jnd\" (UID: \"863f75b6-d5da-405e-a695-be1c0eb08446\") " pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd"
Sep 07 03:52:56.769934 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.769935    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/863f75b6-d5da-405e-a695-be1c0eb08446-webhook-cert\") pod \"opendatahub-operator-controller-manager-756ff4ddd-m2jnd\" (UID: \"863f75b6-d5da-405e-a695-be1c0eb08446\") " pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd"
Sep 07 03:52:56.770385 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.769987    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85dbd\" (UniqueName: \"kubernetes.io/projected/863f75b6-d5da-405e-a695-be1c0eb08446-kube-api-access-85dbd\") pod \"opendatahub-operator-controller-manager-756ff4ddd-m2jnd\" (UID: \"863f75b6-d5da-405e-a695-be1c0eb08446\") " pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd"
Sep 07 03:52:56.770385 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.770022    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/863f75b6-d5da-405e-a695-be1c0eb08446-tmp\") pod \"opendatahub-operator-controller-manager-756ff4ddd-m2jnd\" (UID: \"863f75b6-d5da-405e-a695-be1c0eb08446\") " pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd"
Sep 07 03:52:56.770385 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.770082    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/863f75b6-d5da-405e-a695-be1c0eb08446-manifests\") pod \"opendatahub-operator-controller-manager-756ff4ddd-m2jnd\" (UID: \"863f75b6-d5da-405e-a695-be1c0eb08446\") " pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd"
Sep 07 03:52:56.780393 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.780365    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:56.871085 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.871048    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/863f75b6-d5da-405e-a695-be1c0eb08446-manifests\") pod \"opendatahub-operator-controller-manager-756ff4ddd-m2jnd\" (UID: \"863f75b6-d5da-405e-a695-be1c0eb08446\") " pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd"
Sep 07 03:52:56.871316 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.871115    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/863f75b6-d5da-405e-a695-be1c0eb08446-apiservice-cert\") pod \"opendatahub-operator-controller-manager-756ff4ddd-m2jnd\" (UID: \"863f75b6-d5da-405e-a695-be1c0eb08446\") " pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd"
Sep 07 03:52:56.871316 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.871142    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/863f75b6-d5da-405e-a695-be1c0eb08446-webhook-cert\") pod \"opendatahub-operator-controller-manager-756ff4ddd-m2jnd\" (UID: \"863f75b6-d5da-405e-a695-be1c0eb08446\") " pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd"
Sep 07 03:52:56.871316 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.871167    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85dbd\" (UniqueName: \"kubernetes.io/projected/863f75b6-d5da-405e-a695-be1c0eb08446-kube-api-access-85dbd\") pod \"opendatahub-operator-controller-manager-756ff4ddd-m2jnd\" (UID: \"863f75b6-d5da-405e-a695-be1c0eb08446\") " pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd"
Sep 07 03:52:56.871316 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.871191    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/863f75b6-d5da-405e-a695-be1c0eb08446-tmp\") pod \"opendatahub-operator-controller-manager-756ff4ddd-m2jnd\" (UID: \"863f75b6-d5da-405e-a695-be1c0eb08446\") " pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd"
Sep 07 03:52:56.871532 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.871479    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/863f75b6-d5da-405e-a695-be1c0eb08446-manifests\") pod \"opendatahub-operator-controller-manager-756ff4ddd-m2jnd\" (UID: \"863f75b6-d5da-405e-a695-be1c0eb08446\") " pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd"
Sep 07 03:52:56.873569 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.873543    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/863f75b6-d5da-405e-a695-be1c0eb08446-tmp\") pod \"opendatahub-operator-controller-manager-756ff4ddd-m2jnd\" (UID: \"863f75b6-d5da-405e-a695-be1c0eb08446\") " pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd"
Sep 07 03:52:56.873782 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.873765    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/863f75b6-d5da-405e-a695-be1c0eb08446-webhook-cert\") pod \"opendatahub-operator-controller-manager-756ff4ddd-m2jnd\" (UID: \"863f75b6-d5da-405e-a695-be1c0eb08446\") " pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd"
Sep 07 03:52:56.873825 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.873790    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/863f75b6-d5da-405e-a695-be1c0eb08446-apiservice-cert\") pod \"opendatahub-operator-controller-manager-756ff4ddd-m2jnd\" (UID: \"863f75b6-d5da-405e-a695-be1c0eb08446\") " pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd"
Sep 07 03:52:56.879705 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.879682    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85dbd\" (UniqueName: \"kubernetes.io/projected/863f75b6-d5da-405e-a695-be1c0eb08446-kube-api-access-85dbd\") pod \"opendatahub-operator-controller-manager-756ff4ddd-m2jnd\" (UID: \"863f75b6-d5da-405e-a695-be1c0eb08446\") " pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd"
Sep 07 03:52:56.884382 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.884357    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:56.969205 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.969171    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd"
Sep 07 03:52:56.983582 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:56.983550    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:57.082781 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:57.082745    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:57.186424 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:57.186386    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:57.248000 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:57.247927    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd"]
Sep 07 03:52:57.252970 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:52:57.252936    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod863f75b6_d5da_405e_a695_be1c0eb08446.slice/crio-49a9dd42c7ace19c28a01ac12ebcc9d1b521779b982064ef67d9851ea3f983e8 WatchSource:0}: Error finding container 49a9dd42c7ace19c28a01ac12ebcc9d1b521779b982064ef67d9851ea3f983e8: Status 404 returned error can't find the container with id 49a9dd42c7ace19c28a01ac12ebcc9d1b521779b982064ef67d9851ea3f983e8
Sep 07 03:52:57.254401 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:57.254360    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:57.317333 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:57.317299    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:57.421827 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:57.421791    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:57.524946 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:57.524867    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:57.751376 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:57.751344    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd" event={"ID":"863f75b6-d5da-405e-a695-be1c0eb08446","Type":"ContainerStarted","Data":"49a9dd42c7ace19c28a01ac12ebcc9d1b521779b982064ef67d9851ea3f983e8"}
Sep 07 03:52:58.298084 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:58.298051    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:52:58.396010 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:52:58.395978    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:00.765836 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:00.765797    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd" event={"ID":"863f75b6-d5da-405e-a695-be1c0eb08446","Type":"ContainerStarted","Data":"be6d0b0f9e967da6ed33618638e134004313d5040b944d3c47d0fa90d51e92ba"}
Sep 07 03:53:01.771200 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:01.771113    2583 generic.go:356] "Generic (PLEG): container finished" podID="863f75b6-d5da-405e-a695-be1c0eb08446" containerID="be6d0b0f9e967da6ed33618638e134004313d5040b944d3c47d0fa90d51e92ba" exitCode=0
Sep 07 03:53:01.771611 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:01.771195    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd" event={"ID":"863f75b6-d5da-405e-a695-be1c0eb08446","Type":"ContainerDied","Data":"be6d0b0f9e967da6ed33618638e134004313d5040b944d3c47d0fa90d51e92ba"}
Sep 07 03:53:02.776799 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:02.776763    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd" event={"ID":"863f75b6-d5da-405e-a695-be1c0eb08446","Type":"ContainerStarted","Data":"c9241433f1ae781bff81c1a9a46207e8de465a464cabf0e3fcbd58d34f74688d"}
Sep 07 03:53:02.777256 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:02.776873    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd"
Sep 07 03:53:02.800428 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:02.800373    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd" podStartSLOduration=2.256139473 podStartE2EDuration="6.800356377s" totalImagesPullingTime="3.419977392s" totalInitContainerRuntime="1.124239512s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:52:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:53:02.798672184 +0000 UTC m=+269.732152905" watchObservedRunningTime="2026-09-07 03:53:02.800356377 +0000 UTC m=+269.733837095"
Sep 07 03:53:08.509194 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:08.509156    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:09.665372 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:09.665334    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:13.783358 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:13.783325    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-756ff4ddd-m2jnd"
Sep 07 03:53:14.667612 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:14.667580    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-758b7d97c6-dp9p2"]
Sep 07 03:53:14.676474 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:14.676453    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-758b7d97c6-dp9p2"
Sep 07 03:53:14.679040 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:14.679014    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 07 03:53:14.679254 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:14.679238    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 07 03:53:14.679314 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:14.679293    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 07 03:53:14.680125 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:14.680096    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-df89q\""
Sep 07 03:53:14.680420 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:14.680395    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 07 03:53:14.680529 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:14.680478    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 07 03:53:14.688921 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:14.688899    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-758b7d97c6-dp9p2"]
Sep 07 03:53:14.723149 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:14.723115    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0da7680f-9fb0-46ea-a483-7085afac2772-manager-config\") pod \"lws-controller-manager-758b7d97c6-dp9p2\" (UID: \"0da7680f-9fb0-46ea-a483-7085afac2772\") " pod="openshift-lws-operator/lws-controller-manager-758b7d97c6-dp9p2"
Sep 07 03:53:14.723149 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:14.723146    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5gkp\" (UniqueName: \"kubernetes.io/projected/0da7680f-9fb0-46ea-a483-7085afac2772-kube-api-access-b5gkp\") pod \"lws-controller-manager-758b7d97c6-dp9p2\" (UID: \"0da7680f-9fb0-46ea-a483-7085afac2772\") " pod="openshift-lws-operator/lws-controller-manager-758b7d97c6-dp9p2"
Sep 07 03:53:14.723359 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:14.723183    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0da7680f-9fb0-46ea-a483-7085afac2772-cert\") pod \"lws-controller-manager-758b7d97c6-dp9p2\" (UID: \"0da7680f-9fb0-46ea-a483-7085afac2772\") " pod="openshift-lws-operator/lws-controller-manager-758b7d97c6-dp9p2"
Sep 07 03:53:14.723359 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:14.723254    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0da7680f-9fb0-46ea-a483-7085afac2772-metrics-cert\") pod \"lws-controller-manager-758b7d97c6-dp9p2\" (UID: \"0da7680f-9fb0-46ea-a483-7085afac2772\") " pod="openshift-lws-operator/lws-controller-manager-758b7d97c6-dp9p2"
Sep 07 03:53:14.824364 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:14.824328    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0da7680f-9fb0-46ea-a483-7085afac2772-metrics-cert\") pod \"lws-controller-manager-758b7d97c6-dp9p2\" (UID: \"0da7680f-9fb0-46ea-a483-7085afac2772\") " pod="openshift-lws-operator/lws-controller-manager-758b7d97c6-dp9p2"
Sep 07 03:53:14.824885 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:14.824402    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0da7680f-9fb0-46ea-a483-7085afac2772-manager-config\") pod \"lws-controller-manager-758b7d97c6-dp9p2\" (UID: \"0da7680f-9fb0-46ea-a483-7085afac2772\") " pod="openshift-lws-operator/lws-controller-manager-758b7d97c6-dp9p2"
Sep 07 03:53:14.824885 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:14.824428    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5gkp\" (UniqueName: \"kubernetes.io/projected/0da7680f-9fb0-46ea-a483-7085afac2772-kube-api-access-b5gkp\") pod \"lws-controller-manager-758b7d97c6-dp9p2\" (UID: \"0da7680f-9fb0-46ea-a483-7085afac2772\") " pod="openshift-lws-operator/lws-controller-manager-758b7d97c6-dp9p2"
Sep 07 03:53:14.824885 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:14.824491    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0da7680f-9fb0-46ea-a483-7085afac2772-cert\") pod \"lws-controller-manager-758b7d97c6-dp9p2\" (UID: \"0da7680f-9fb0-46ea-a483-7085afac2772\") " pod="openshift-lws-operator/lws-controller-manager-758b7d97c6-dp9p2"
Sep 07 03:53:14.825085 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:14.825067    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0da7680f-9fb0-46ea-a483-7085afac2772-manager-config\") pod \"lws-controller-manager-758b7d97c6-dp9p2\" (UID: \"0da7680f-9fb0-46ea-a483-7085afac2772\") " pod="openshift-lws-operator/lws-controller-manager-758b7d97c6-dp9p2"
Sep 07 03:53:14.826817 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:14.826796    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0da7680f-9fb0-46ea-a483-7085afac2772-metrics-cert\") pod \"lws-controller-manager-758b7d97c6-dp9p2\" (UID: \"0da7680f-9fb0-46ea-a483-7085afac2772\") " pod="openshift-lws-operator/lws-controller-manager-758b7d97c6-dp9p2"
Sep 07 03:53:14.826932 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:14.826842    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0da7680f-9fb0-46ea-a483-7085afac2772-cert\") pod \"lws-controller-manager-758b7d97c6-dp9p2\" (UID: \"0da7680f-9fb0-46ea-a483-7085afac2772\") " pod="openshift-lws-operator/lws-controller-manager-758b7d97c6-dp9p2"
Sep 07 03:53:14.833400 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:14.833377    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5gkp\" (UniqueName: \"kubernetes.io/projected/0da7680f-9fb0-46ea-a483-7085afac2772-kube-api-access-b5gkp\") pod \"lws-controller-manager-758b7d97c6-dp9p2\" (UID: \"0da7680f-9fb0-46ea-a483-7085afac2772\") " pod="openshift-lws-operator/lws-controller-manager-758b7d97c6-dp9p2"
Sep 07 03:53:14.986852 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:14.986819    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-758b7d97c6-dp9p2"
Sep 07 03:53:15.263859 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:15.263835    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-758b7d97c6-dp9p2"]
Sep 07 03:53:15.266335 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:53:15.266313    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0da7680f_9fb0_46ea_a483_7085afac2772.slice/crio-74687b255ced6caef47ba2ea15fc1ca74db03f71169bba64da5ed7141b5e13ee WatchSource:0}: Error finding container 74687b255ced6caef47ba2ea15fc1ca74db03f71169bba64da5ed7141b5e13ee: Status 404 returned error can't find the container with id 74687b255ced6caef47ba2ea15fc1ca74db03f71169bba64da5ed7141b5e13ee
Sep 07 03:53:15.787385 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:15.787352    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:15.830598 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:15.830565    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-758b7d97c6-dp9p2" event={"ID":"0da7680f-9fb0-46ea-a483-7085afac2772","Type":"ContainerStarted","Data":"74687b255ced6caef47ba2ea15fc1ca74db03f71169bba64da5ed7141b5e13ee"}
Sep 07 03:53:15.922379 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:15.922345    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:17.466717 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:17.466686    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:17.532352 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:17.532321    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-59d7d88d64-m5zd6"]
Sep 07 03:53:17.536269 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:17.536247    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-59d7d88d64-m5zd6"
Sep 07 03:53:17.538578 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:17.538554    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 07 03:53:17.538700 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:17.538688    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 07 03:53:17.538762 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:17.538732    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-d46lm\""
Sep 07 03:53:17.539686 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:17.539666    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 07 03:53:17.539686 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:17.539678    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 07 03:53:17.547262 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:17.547239    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w7st\" (UniqueName: \"kubernetes.io/projected/f19fce4a-64b9-4988-b9bc-08f42b5c8843-kube-api-access-2w7st\") pod \"kube-auth-proxy-59d7d88d64-m5zd6\" (UID: \"f19fce4a-64b9-4988-b9bc-08f42b5c8843\") " pod="openshift-ingress/kube-auth-proxy-59d7d88d64-m5zd6"
Sep 07 03:53:17.547568 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:17.547539    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f19fce4a-64b9-4988-b9bc-08f42b5c8843-tls-certs\") pod \"kube-auth-proxy-59d7d88d64-m5zd6\" (UID: \"f19fce4a-64b9-4988-b9bc-08f42b5c8843\") " pod="openshift-ingress/kube-auth-proxy-59d7d88d64-m5zd6"
Sep 07 03:53:17.547671 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:17.547621    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f19fce4a-64b9-4988-b9bc-08f42b5c8843-tmp\") pod \"kube-auth-proxy-59d7d88d64-m5zd6\" (UID: \"f19fce4a-64b9-4988-b9bc-08f42b5c8843\") " pod="openshift-ingress/kube-auth-proxy-59d7d88d64-m5zd6"
Sep 07 03:53:17.547744 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:17.547723    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-59d7d88d64-m5zd6"]
Sep 07 03:53:17.648856 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:17.648824    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f19fce4a-64b9-4988-b9bc-08f42b5c8843-tls-certs\") pod \"kube-auth-proxy-59d7d88d64-m5zd6\" (UID: \"f19fce4a-64b9-4988-b9bc-08f42b5c8843\") " pod="openshift-ingress/kube-auth-proxy-59d7d88d64-m5zd6"
Sep 07 03:53:17.649056 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:17.648865    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f19fce4a-64b9-4988-b9bc-08f42b5c8843-tmp\") pod \"kube-auth-proxy-59d7d88d64-m5zd6\" (UID: \"f19fce4a-64b9-4988-b9bc-08f42b5c8843\") " pod="openshift-ingress/kube-auth-proxy-59d7d88d64-m5zd6"
Sep 07 03:53:17.649056 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:17.648938    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w7st\" (UniqueName: \"kubernetes.io/projected/f19fce4a-64b9-4988-b9bc-08f42b5c8843-kube-api-access-2w7st\") pod \"kube-auth-proxy-59d7d88d64-m5zd6\" (UID: \"f19fce4a-64b9-4988-b9bc-08f42b5c8843\") " pod="openshift-ingress/kube-auth-proxy-59d7d88d64-m5zd6"
Sep 07 03:53:17.651736 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:17.651703    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f19fce4a-64b9-4988-b9bc-08f42b5c8843-tmp\") pod \"kube-auth-proxy-59d7d88d64-m5zd6\" (UID: \"f19fce4a-64b9-4988-b9bc-08f42b5c8843\") " pod="openshift-ingress/kube-auth-proxy-59d7d88d64-m5zd6"
Sep 07 03:53:17.651911 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:17.651894    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f19fce4a-64b9-4988-b9bc-08f42b5c8843-tls-certs\") pod \"kube-auth-proxy-59d7d88d64-m5zd6\" (UID: \"f19fce4a-64b9-4988-b9bc-08f42b5c8843\") " pod="openshift-ingress/kube-auth-proxy-59d7d88d64-m5zd6"
Sep 07 03:53:17.657957 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:17.657931    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w7st\" (UniqueName: \"kubernetes.io/projected/f19fce4a-64b9-4988-b9bc-08f42b5c8843-kube-api-access-2w7st\") pod \"kube-auth-proxy-59d7d88d64-m5zd6\" (UID: \"f19fce4a-64b9-4988-b9bc-08f42b5c8843\") " pod="openshift-ingress/kube-auth-proxy-59d7d88d64-m5zd6"
Sep 07 03:53:17.694206 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:17.694173    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:17.840180 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:17.840150    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-758b7d97c6-dp9p2" event={"ID":"0da7680f-9fb0-46ea-a483-7085afac2772","Type":"ContainerStarted","Data":"a3e208001baa9c78041f3d722d7fdccb26ce7506c4f8152ba0ae7af8d18b4c58"}
Sep 07 03:53:17.840369 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:17.840281    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-758b7d97c6-dp9p2"
Sep 07 03:53:17.849525 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:17.849499    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-59d7d88d64-m5zd6"
Sep 07 03:53:17.858265 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:17.858197    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-758b7d97c6-dp9p2" podStartSLOduration=1.372091507 podStartE2EDuration="3.85818287s" totalImagesPullingTime="2.486091363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:53:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:53:17.856364854 +0000 UTC m=+284.789845573" watchObservedRunningTime="2026-09-07 03:53:17.85818287 +0000 UTC m=+284.791663587"
Sep 07 03:53:18.126557 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:18.126529    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-59d7d88d64-m5zd6"]
Sep 07 03:53:18.129283 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:53:18.129254    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf19fce4a_64b9_4988_b9bc_08f42b5c8843.slice/crio-b5c807828d87518737b8a786898a6cee2b703299d8104957a2ef04099448ecf5 WatchSource:0}: Error finding container b5c807828d87518737b8a786898a6cee2b703299d8104957a2ef04099448ecf5: Status 404 returned error can't find the container with id b5c807828d87518737b8a786898a6cee2b703299d8104957a2ef04099448ecf5
Sep 07 03:53:18.585280 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:18.585223    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:18.684123 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:18.684082    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:18.848350 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:18.848251    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-59d7d88d64-m5zd6" event={"ID":"f19fce4a-64b9-4988-b9bc-08f42b5c8843","Type":"ContainerStarted","Data":"b5c807828d87518737b8a786898a6cee2b703299d8104957a2ef04099448ecf5"}
Sep 07 03:53:19.487650 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:19.487610    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:21.571931 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:21.571897    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:21.672354 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:21.672317    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:21.862688 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:21.862595    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-59d7d88d64-m5zd6" event={"ID":"f19fce4a-64b9-4988-b9bc-08f42b5c8843","Type":"ContainerStarted","Data":"224c220d3e857c61bbf1c7dac6b1d23a3f252bcc126894d62ed6220c13490cbe"}
Sep 07 03:53:21.879512 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:21.879453    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-59d7d88d64-m5zd6" podStartSLOduration=2.1555062879999998 podStartE2EDuration="4.879436583s" totalImagesPullingTime="2.723930295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:53:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:53:21.87850417 +0000 UTC m=+288.811984888" watchObservedRunningTime="2026-09-07 03:53:21.879436583 +0000 UTC m=+288.812917322"
Sep 07 03:53:23.377464 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:23.377426    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:23.475273 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:23.475192    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:23.668938 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:23.668842    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:23.810720 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:23.810689    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:23.934458 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:23.934432    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:27.117400 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:27.117360    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:27.218766 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:27.218735    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:27.318960 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:27.318923    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:27.520658 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:27.520623    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:27.657664 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:27.657634    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:27.774702 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:27.774619    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:28.851009 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:28.850979    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-758b7d97c6-dp9p2"
Sep 07 03:53:33.439898 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:33.439873    2583 kubelet.go:1648] "Image garbage collection succeeded"
Sep 07 03:53:37.446458 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:37.446418    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:37.572848 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:37.572813    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-69hxm"]
Sep 07 03:53:37.574986 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:37.574971    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-69hxm"
Sep 07 03:53:37.588642 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:37.588616    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 07 03:53:37.588642 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:37.588638    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-chh9r\""
Sep 07 03:53:37.603858 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:37.603820    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-69hxm"]
Sep 07 03:53:37.620758 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:37.620727    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:37.623841 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:37.623818    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4g68\" (UniqueName: \"kubernetes.io/projected/ef077a47-6929-4638-8d5a-7efdd731a387-kube-api-access-r4g68\") pod \"llmisvc-controller-manager-777c6bd7d5-69hxm\" (UID: \"ef077a47-6929-4638-8d5a-7efdd731a387\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-69hxm"
Sep 07 03:53:37.623954 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:37.623859    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef077a47-6929-4638-8d5a-7efdd731a387-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-69hxm\" (UID: \"ef077a47-6929-4638-8d5a-7efdd731a387\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-69hxm"
Sep 07 03:53:37.725140 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:37.725045    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4g68\" (UniqueName: \"kubernetes.io/projected/ef077a47-6929-4638-8d5a-7efdd731a387-kube-api-access-r4g68\") pod \"llmisvc-controller-manager-777c6bd7d5-69hxm\" (UID: \"ef077a47-6929-4638-8d5a-7efdd731a387\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-69hxm"
Sep 07 03:53:37.725140 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:37.725089    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef077a47-6929-4638-8d5a-7efdd731a387-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-69hxm\" (UID: \"ef077a47-6929-4638-8d5a-7efdd731a387\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-69hxm"
Sep 07 03:53:37.727735 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:37.727711    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef077a47-6929-4638-8d5a-7efdd731a387-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-69hxm\" (UID: \"ef077a47-6929-4638-8d5a-7efdd731a387\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-69hxm"
Sep 07 03:53:37.735095 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:37.735068    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4g68\" (UniqueName: \"kubernetes.io/projected/ef077a47-6929-4638-8d5a-7efdd731a387-kube-api-access-r4g68\") pod \"llmisvc-controller-manager-777c6bd7d5-69hxm\" (UID: \"ef077a47-6929-4638-8d5a-7efdd731a387\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-69hxm"
Sep 07 03:53:37.745534 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:37.745510    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:37.884771 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:37.884734    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-69hxm"
Sep 07 03:53:37.938108 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:37.938071    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:38.155367 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:38.155277    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-69hxm"]
Sep 07 03:53:38.157775 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:53:38.157745    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podef077a47_6929_4638_8d5a_7efdd731a387.slice/crio-5d89f6895dfb172eda9f7c47eeffdcb0c278a3904c041cfc121630e1e94c2ac2 WatchSource:0}: Error finding container 5d89f6895dfb172eda9f7c47eeffdcb0c278a3904c041cfc121630e1e94c2ac2: Status 404 returned error can't find the container with id 5d89f6895dfb172eda9f7c47eeffdcb0c278a3904c041cfc121630e1e94c2ac2
Sep 07 03:53:38.159163 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:38.159145    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 03:53:38.850125 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:38.850093    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:38.942876 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:38.942845    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-69hxm" event={"ID":"ef077a47-6929-4638-8d5a-7efdd731a387","Type":"ContainerStarted","Data":"5d89f6895dfb172eda9f7c47eeffdcb0c278a3904c041cfc121630e1e94c2ac2"}
Sep 07 03:53:38.946499 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:38.946466    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:39.048028 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:39.047993    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:40.954414 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:40.954385    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-69hxm" event={"ID":"ef077a47-6929-4638-8d5a-7efdd731a387","Type":"ContainerStarted","Data":"ca4473ce38faf718ac74c6621ce9c1a2fec15c4a37b87f1094f0654bdc13baee"}
Sep 07 03:53:40.954773 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:40.954445    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-69hxm"
Sep 07 03:53:40.972021 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:40.971973    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-69hxm" podStartSLOduration=1.347305252 podStartE2EDuration="3.971957517s" totalImagesPullingTime="2.624652265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:53:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:53:40.970764863 +0000 UTC m=+307.904245581" watchObservedRunningTime="2026-09-07 03:53:40.971957517 +0000 UTC m=+307.905438270"
Sep 07 03:53:45.393848 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:45.393801    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:45.454239 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:45.454181    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:53:47.146066 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:53:47.146031    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:11.961609 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:11.961578    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-69hxm"
Sep 07 03:54:22.851667 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:22.851635    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:27.919232 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:27.919147    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:27.989787 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:27.989751    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:34.233943 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:34.233909    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:34.335490 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:34.335457    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:34.435754 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:34.435714    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:34.965909 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:34.965865    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:40.608582 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:40.608545    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:41.407401 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:41.407374    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:42.219476 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:42.219443    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:44.491707 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:44.491674    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:44.588805 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:44.588776    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:45.334466 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:45.334431    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:45.438579 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:45.438542    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:45.533401 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:45.533369    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:45.817312 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:45.817282    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:46.078408 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:46.078329    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:48.415689 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:48.415648    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:48.522391 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:48.522361    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:50.645129 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:50.645091    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:50.855567 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:50.855532    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:52.416084 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:52.416051    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-2kdgv"]
Sep 07 03:54:52.419748 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:52.419724    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-2kdgv"
Sep 07 03:54:52.422489 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:52.422466    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 07 03:54:52.422628 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:52.422512    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 07 03:54:52.422687 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:52.422630    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-m9vpk\""
Sep 07 03:54:52.422736 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:52.422687    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 07 03:54:52.436163 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:52.436138    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-2kdgv"]
Sep 07 03:54:52.473924 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:52.473886    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:52.554693 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:52.554654    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqn9v\" (UniqueName: \"kubernetes.io/projected/03893c8b-3664-45ec-9df3-056fd805cdcb-kube-api-access-sqn9v\") pod \"dns-operator-controller-manager-5985564776-2kdgv\" (UID: \"03893c8b-3664-45ec-9df3-056fd805cdcb\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-2kdgv"
Sep 07 03:54:52.576252 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:52.576187    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:52.655656 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:52.655624    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqn9v\" (UniqueName: \"kubernetes.io/projected/03893c8b-3664-45ec-9df3-056fd805cdcb-kube-api-access-sqn9v\") pod \"dns-operator-controller-manager-5985564776-2kdgv\" (UID: \"03893c8b-3664-45ec-9df3-056fd805cdcb\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-2kdgv"
Sep 07 03:54:52.665798 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:52.665759    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqn9v\" (UniqueName: \"kubernetes.io/projected/03893c8b-3664-45ec-9df3-056fd805cdcb-kube-api-access-sqn9v\") pod \"dns-operator-controller-manager-5985564776-2kdgv\" (UID: \"03893c8b-3664-45ec-9df3-056fd805cdcb\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-2kdgv"
Sep 07 03:54:52.730527 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:52.730488    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-2kdgv"
Sep 07 03:54:53.004433 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:53.004352    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-2kdgv"]
Sep 07 03:54:53.007773 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:54:53.007743    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03893c8b_3664_45ec_9df3_056fd805cdcb.slice/crio-2f9941314f55a41d419e43f513253d28e6c80e56f94150d49f43ce99b475377e WatchSource:0}: Error finding container 2f9941314f55a41d419e43f513253d28e6c80e56f94150d49f43ce99b475377e: Status 404 returned error can't find the container with id 2f9941314f55a41d419e43f513253d28e6c80e56f94150d49f43ce99b475377e
Sep 07 03:54:53.262759 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:53.262673    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-2kdgv" event={"ID":"03893c8b-3664-45ec-9df3-056fd805cdcb","Type":"ContainerStarted","Data":"2f9941314f55a41d419e43f513253d28e6c80e56f94150d49f43ce99b475377e"}
Sep 07 03:54:55.076913 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:55.076874    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-n7x9r"]
Sep 07 03:54:55.079451 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:55.079419    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:55.083856 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:55.083831    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-n7x9r"
Sep 07 03:54:55.087029 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:55.086979    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 07 03:54:55.087542 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:55.087339    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-l4lt8\""
Sep 07 03:54:55.087542 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:55.087417    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 07 03:54:55.093983 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:55.093945    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-n7x9r"]
Sep 07 03:54:55.177796 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:55.177742    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/05742da4-d5c2-4b32-b7ca-e2e0e66dfb60-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-n7x9r\" (UID: \"05742da4-d5c2-4b32-b7ca-e2e0e66dfb60\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-n7x9r"
Sep 07 03:54:55.177963 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:55.177839    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-788m5\" (UniqueName: \"kubernetes.io/projected/05742da4-d5c2-4b32-b7ca-e2e0e66dfb60-kube-api-access-788m5\") pod \"kuadrant-console-plugin-598f86468b-n7x9r\" (UID: \"05742da4-d5c2-4b32-b7ca-e2e0e66dfb60\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-n7x9r"
Sep 07 03:54:55.177963 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:55.177870    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/05742da4-d5c2-4b32-b7ca-e2e0e66dfb60-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-n7x9r\" (UID: \"05742da4-d5c2-4b32-b7ca-e2e0e66dfb60\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-n7x9r"
Sep 07 03:54:55.186159 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:55.186124    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:55.279262 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:55.279226    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-788m5\" (UniqueName: \"kubernetes.io/projected/05742da4-d5c2-4b32-b7ca-e2e0e66dfb60-kube-api-access-788m5\") pod \"kuadrant-console-plugin-598f86468b-n7x9r\" (UID: \"05742da4-d5c2-4b32-b7ca-e2e0e66dfb60\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-n7x9r"
Sep 07 03:54:55.279455 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:55.279279    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/05742da4-d5c2-4b32-b7ca-e2e0e66dfb60-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-n7x9r\" (UID: \"05742da4-d5c2-4b32-b7ca-e2e0e66dfb60\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-n7x9r"
Sep 07 03:54:55.279455 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:55.279350    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/05742da4-d5c2-4b32-b7ca-e2e0e66dfb60-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-n7x9r\" (UID: \"05742da4-d5c2-4b32-b7ca-e2e0e66dfb60\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-n7x9r"
Sep 07 03:54:55.280118 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:55.280093    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/05742da4-d5c2-4b32-b7ca-e2e0e66dfb60-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-n7x9r\" (UID: \"05742da4-d5c2-4b32-b7ca-e2e0e66dfb60\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-n7x9r"
Sep 07 03:54:55.282293 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:55.282264    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/05742da4-d5c2-4b32-b7ca-e2e0e66dfb60-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-n7x9r\" (UID: \"05742da4-d5c2-4b32-b7ca-e2e0e66dfb60\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-n7x9r"
Sep 07 03:54:55.288375 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:55.288349    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-788m5\" (UniqueName: \"kubernetes.io/projected/05742da4-d5c2-4b32-b7ca-e2e0e66dfb60-kube-api-access-788m5\") pod \"kuadrant-console-plugin-598f86468b-n7x9r\" (UID: \"05742da4-d5c2-4b32-b7ca-e2e0e66dfb60\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-n7x9r"
Sep 07 03:54:55.401601 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:55.401515    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-n7x9r"
Sep 07 03:54:55.414549 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:55.414517    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:55.483406 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:55.483374    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:55.757528 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:55.757500    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-n7x9r"]
Sep 07 03:54:55.759925 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:54:55.759901    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05742da4_d5c2_4b32_b7ca_e2e0e66dfb60.slice/crio-b4a2bbf6e374e102673e6d3b4ae3e47c29ae74dacb911bfc1851673e8ba50968 WatchSource:0}: Error finding container b4a2bbf6e374e102673e6d3b4ae3e47c29ae74dacb911bfc1851673e8ba50968: Status 404 returned error can't find the container with id b4a2bbf6e374e102673e6d3b4ae3e47c29ae74dacb911bfc1851673e8ba50968
Sep 07 03:54:55.819145 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:55.819115    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:55.919515 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:55.919477    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:56.217641 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:56.217611    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:56.272241 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:56.272189    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:54:56.285068 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:56.285029    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-n7x9r" event={"ID":"05742da4-d5c2-4b32-b7ca-e2e0e66dfb60","Type":"ContainerStarted","Data":"b4a2bbf6e374e102673e6d3b4ae3e47c29ae74dacb911bfc1851673e8ba50968"}
Sep 07 03:54:56.286759 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:56.286734    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-2kdgv" event={"ID":"03893c8b-3664-45ec-9df3-056fd805cdcb","Type":"ContainerStarted","Data":"d6b80e78f74e347227405ed9df0b6a372fe6d59b27781cb9593504e666dd3aaa"}
Sep 07 03:54:56.286922 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:56.286904    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-2kdgv"
Sep 07 03:54:56.306634 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:54:56.306587    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-2kdgv" podStartSLOduration=1.75129315 podStartE2EDuration="4.306564634s" totalImagesPullingTime="2.555271484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:54:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:54:56.305836368 +0000 UTC m=+383.239317086" watchObservedRunningTime="2026-09-07 03:54:56.306564634 +0000 UTC m=+383.240045355"
Sep 07 03:55:01.314724 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:01.314690    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-n7x9r" event={"ID":"05742da4-d5c2-4b32-b7ca-e2e0e66dfb60","Type":"ContainerStarted","Data":"970df8b0989036fe3789ab557f67ec993be57458a96816e31561f511d3e73638"}
Sep 07 03:55:01.332778 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:01.332728    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-n7x9r" podStartSLOduration=1.800062571 podStartE2EDuration="6.332713029s" totalImagesPullingTime="4.532650458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:54:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:55:01.331387019 +0000 UTC m=+388.264867744" watchObservedRunningTime="2026-09-07 03:55:01.332713029 +0000 UTC m=+388.266193748"
Sep 07 03:55:04.239624 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:04.239580    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:04.351182 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:04.351142    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:07.247726 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:07.247678    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:07.294792 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:07.294758    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-2kdgv"
Sep 07 03:55:07.332225 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:07.332164    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:09.699749 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:09.699715    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:09.799588 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:09.799537    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:16.629208 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:16.629177    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:16.730278 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:16.730244    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:17.053344 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:17.053307    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:17.123027 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:17.122996    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:17.232521 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:17.232486    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:17.333070 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:17.332968    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:20.546345 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:20.546316    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:20.604123 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:20.604091    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:26.035190 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:26.035161    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:26.135586 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:26.135553    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:29.036167 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:29.036129    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:29.137924 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:29.137847    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:31.493838 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:31.493808    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:31.551894 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:31.551860    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:35.283645 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:35.283609    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:35.495165 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:35.495132    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:36.849274 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:36.849247    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:36.900654 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:36.900621    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:40.219565 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:40.219529    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-zc9nf"]
Sep 07 03:55:40.226192 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:40.226170    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-zc9nf"
Sep 07 03:55:40.228767 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:40.228742    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 07 03:55:40.233424 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:40.233406    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-zc9nf"]
Sep 07 03:55:40.251507 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:40.251474    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:40.322854 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:40.322809    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:40.327828 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:40.327803    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-zc9nf"]
Sep 07 03:55:40.362744 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:40.362707    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3e87d2b-16fe-4cde-8d01-feea42bf7f94-config-file\") pod \"limitador-limitador-859c4d96d6-zc9nf\" (UID: \"f3e87d2b-16fe-4cde-8d01-feea42bf7f94\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-zc9nf"
Sep 07 03:55:40.362910 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:40.362755    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhtdv\" (UniqueName: \"kubernetes.io/projected/f3e87d2b-16fe-4cde-8d01-feea42bf7f94-kube-api-access-fhtdv\") pod \"limitador-limitador-859c4d96d6-zc9nf\" (UID: \"f3e87d2b-16fe-4cde-8d01-feea42bf7f94\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-zc9nf"
Sep 07 03:55:40.390488 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:40.390441    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:40.464171 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:40.464140    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3e87d2b-16fe-4cde-8d01-feea42bf7f94-config-file\") pod \"limitador-limitador-859c4d96d6-zc9nf\" (UID: \"f3e87d2b-16fe-4cde-8d01-feea42bf7f94\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-zc9nf"
Sep 07 03:55:40.464368 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:40.464184    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhtdv\" (UniqueName: \"kubernetes.io/projected/f3e87d2b-16fe-4cde-8d01-feea42bf7f94-kube-api-access-fhtdv\") pod \"limitador-limitador-859c4d96d6-zc9nf\" (UID: \"f3e87d2b-16fe-4cde-8d01-feea42bf7f94\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-zc9nf"
Sep 07 03:55:40.464818 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:40.464795    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3e87d2b-16fe-4cde-8d01-feea42bf7f94-config-file\") pod \"limitador-limitador-859c4d96d6-zc9nf\" (UID: \"f3e87d2b-16fe-4cde-8d01-feea42bf7f94\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-zc9nf"
Sep 07 03:55:40.472476 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:40.472421    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhtdv\" (UniqueName: \"kubernetes.io/projected/f3e87d2b-16fe-4cde-8d01-feea42bf7f94-kube-api-access-fhtdv\") pod \"limitador-limitador-859c4d96d6-zc9nf\" (UID: \"f3e87d2b-16fe-4cde-8d01-feea42bf7f94\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-zc9nf"
Sep 07 03:55:40.536911 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:40.536874    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-zc9nf"
Sep 07 03:55:40.806185 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:40.806103    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-zc9nf"]
Sep 07 03:55:40.809398 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:55:40.809366    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3e87d2b_16fe_4cde_8d01_feea42bf7f94.slice/crio-04aa18d21c08b70359b9d730dd47eb07df99f172a5806f946a884487582124f0 WatchSource:0}: Error finding container 04aa18d21c08b70359b9d730dd47eb07df99f172a5806f946a884487582124f0: Status 404 returned error can't find the container with id 04aa18d21c08b70359b9d730dd47eb07df99f172a5806f946a884487582124f0
Sep 07 03:55:41.047173 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:41.047139    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:41.148369 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:41.148285    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:41.161358 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:41.161325    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-bt56l"]
Sep 07 03:55:41.164204 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:41.164188    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-bt56l"
Sep 07 03:55:41.166550 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:41.166530    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-mws5h\""
Sep 07 03:55:41.169366 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:41.169347    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-bt56l"]
Sep 07 03:55:41.271602 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:41.271574    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2nxp\" (UniqueName: \"kubernetes.io/projected/e7e64784-85d0-408d-8d5f-5e9294e8a09c-kube-api-access-c2nxp\") pod \"authorino-c548b5d7d-bt56l\" (UID: \"e7e64784-85d0-408d-8d5f-5e9294e8a09c\") " pod="kuadrant-system/authorino-c548b5d7d-bt56l"
Sep 07 03:55:41.372615 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:41.372579    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2nxp\" (UniqueName: \"kubernetes.io/projected/e7e64784-85d0-408d-8d5f-5e9294e8a09c-kube-api-access-c2nxp\") pod \"authorino-c548b5d7d-bt56l\" (UID: \"e7e64784-85d0-408d-8d5f-5e9294e8a09c\") " pod="kuadrant-system/authorino-c548b5d7d-bt56l"
Sep 07 03:55:41.381262 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:41.381235    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2nxp\" (UniqueName: \"kubernetes.io/projected/e7e64784-85d0-408d-8d5f-5e9294e8a09c-kube-api-access-c2nxp\") pod \"authorino-c548b5d7d-bt56l\" (UID: \"e7e64784-85d0-408d-8d5f-5e9294e8a09c\") " pod="kuadrant-system/authorino-c548b5d7d-bt56l"
Sep 07 03:55:41.474185 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:41.474157    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-bt56l"
Sep 07 03:55:41.492285 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:41.492257    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-zc9nf" event={"ID":"f3e87d2b-16fe-4cde-8d01-feea42bf7f94","Type":"ContainerStarted","Data":"04aa18d21c08b70359b9d730dd47eb07df99f172a5806f946a884487582124f0"}
Sep 07 03:55:41.742360 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:41.742287    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-bt56l"]
Sep 07 03:55:41.745338 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:55:41.745305    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7e64784_85d0_408d_8d5f_5e9294e8a09c.slice/crio-15214fcadaadd65f780d1158aa05fa2b5600dc02354aae37bbeb7a7e11997e9b WatchSource:0}: Error finding container 15214fcadaadd65f780d1158aa05fa2b5600dc02354aae37bbeb7a7e11997e9b: Status 404 returned error can't find the container with id 15214fcadaadd65f780d1158aa05fa2b5600dc02354aae37bbeb7a7e11997e9b
Sep 07 03:55:42.497787 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:42.497689    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-zc9nf" event={"ID":"f3e87d2b-16fe-4cde-8d01-feea42bf7f94","Type":"ContainerStarted","Data":"bebe1a4b37667bc63f76d6ae46b3d0c554da933e3c56d67e7351054e883767e0"}
Sep 07 03:55:42.497787 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:42.497749    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-zc9nf"
Sep 07 03:55:42.498890 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:42.498870    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-bt56l" event={"ID":"e7e64784-85d0-408d-8d5f-5e9294e8a09c","Type":"ContainerStarted","Data":"15214fcadaadd65f780d1158aa05fa2b5600dc02354aae37bbeb7a7e11997e9b"}
Sep 07 03:55:42.514953 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:42.514908    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-zc9nf" podStartSLOduration=1.093247233 podStartE2EDuration="2.51489287s" totalImagesPullingTime="1.421645637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:55:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:55:42.513729875 +0000 UTC m=+429.447210597" watchObservedRunningTime="2026-09-07 03:55:42.51489287 +0000 UTC m=+429.448373588"
Sep 07 03:55:44.509677 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:44.509638    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-bt56l" event={"ID":"e7e64784-85d0-408d-8d5f-5e9294e8a09c","Type":"ContainerStarted","Data":"1e662722d880de7cdc2c477ac58a772e5ac6728317a03443b36c5153c7ab9437"}
Sep 07 03:55:44.526033 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:44.525991    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-bt56l" podStartSLOduration=1.299400861 podStartE2EDuration="3.525974701s" totalImagesPullingTime="2.22657384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:55:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:55:44.525167745 +0000 UTC m=+431.458648467" watchObservedRunningTime="2026-09-07 03:55:44.525974701 +0000 UTC m=+431.459455444"
Sep 07 03:55:46.223979 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:46.223947    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:53.505512 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:53.505469    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-zc9nf"
Sep 07 03:55:54.188885 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:54.188849    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-zc9nf"]
Sep 07 03:55:54.189064 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:54.189042    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-zc9nf" podUID="f3e87d2b-16fe-4cde-8d01-feea42bf7f94" containerName="limitador" containerID="cri-o://bebe1a4b37667bc63f76d6ae46b3d0c554da933e3c56d67e7351054e883767e0" gracePeriod=30
Sep 07 03:55:54.553983 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:54.553913    2583 generic.go:356] "Generic (PLEG): container finished" podID="f3e87d2b-16fe-4cde-8d01-feea42bf7f94" containerID="bebe1a4b37667bc63f76d6ae46b3d0c554da933e3c56d67e7351054e883767e0" exitCode=0
Sep 07 03:55:54.553983 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:54.553959    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-zc9nf" event={"ID":"f3e87d2b-16fe-4cde-8d01-feea42bf7f94","Type":"ContainerDied","Data":"bebe1a4b37667bc63f76d6ae46b3d0c554da933e3c56d67e7351054e883767e0"}
Sep 07 03:55:55.131939 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:55.131912    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-zc9nf"
Sep 07 03:55:55.303618 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:55.303526    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhtdv\" (UniqueName: \"kubernetes.io/projected/f3e87d2b-16fe-4cde-8d01-feea42bf7f94-kube-api-access-fhtdv\") pod \"f3e87d2b-16fe-4cde-8d01-feea42bf7f94\" (UID: \"f3e87d2b-16fe-4cde-8d01-feea42bf7f94\") "
Sep 07 03:55:55.303618 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:55.303583    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3e87d2b-16fe-4cde-8d01-feea42bf7f94-config-file\") pod \"f3e87d2b-16fe-4cde-8d01-feea42bf7f94\" (UID: \"f3e87d2b-16fe-4cde-8d01-feea42bf7f94\") "
Sep 07 03:55:55.303990 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:55.303966    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3e87d2b-16fe-4cde-8d01-feea42bf7f94-config-file" (OuterVolumeSpecName: "config-file") pod "f3e87d2b-16fe-4cde-8d01-feea42bf7f94" (UID: "f3e87d2b-16fe-4cde-8d01-feea42bf7f94"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 03:55:55.305855 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:55.305830    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3e87d2b-16fe-4cde-8d01-feea42bf7f94-kube-api-access-fhtdv" (OuterVolumeSpecName: "kube-api-access-fhtdv") pod "f3e87d2b-16fe-4cde-8d01-feea42bf7f94" (UID: "f3e87d2b-16fe-4cde-8d01-feea42bf7f94"). InnerVolumeSpecName "kube-api-access-fhtdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 03:55:55.404383 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:55.404348    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhtdv\" (UniqueName: \"kubernetes.io/projected/f3e87d2b-16fe-4cde-8d01-feea42bf7f94-kube-api-access-fhtdv\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:55:55.404383 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:55.404377    2583 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3e87d2b-16fe-4cde-8d01-feea42bf7f94-config-file\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:55:55.559877 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:55.559804    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-zc9nf"
Sep 07 03:55:55.560285 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:55.559801    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-zc9nf" event={"ID":"f3e87d2b-16fe-4cde-8d01-feea42bf7f94","Type":"ContainerDied","Data":"04aa18d21c08b70359b9d730dd47eb07df99f172a5806f946a884487582124f0"}
Sep 07 03:55:55.560285 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:55.559938    2583 scope.go:117] "RemoveContainer" containerID="bebe1a4b37667bc63f76d6ae46b3d0c554da933e3c56d67e7351054e883767e0"
Sep 07 03:55:55.577784 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:55.577756    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-zc9nf"]
Sep 07 03:55:55.581621 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:55.581595    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-zc9nf"]
Sep 07 03:55:56.348751 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:56.348717    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:56.448130 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:56.448094    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:56.802093 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:56.802062    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ztxnh"]
Sep 07 03:55:56.802591 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:56.802573    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3e87d2b-16fe-4cde-8d01-feea42bf7f94" containerName="limitador"
Sep 07 03:55:56.802591 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:56.802590    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3e87d2b-16fe-4cde-8d01-feea42bf7f94" containerName="limitador"
Sep 07 03:55:56.802696 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:56.802664    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3e87d2b-16fe-4cde-8d01-feea42bf7f94" containerName="limitador"
Sep 07 03:55:56.806945 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:56.806927    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ztxnh"
Sep 07 03:55:56.809769 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:56.809748    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 07 03:55:56.809898 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:56.809780    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 07 03:55:56.810656 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:56.810642    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 03:55:56.810758 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:56.810645    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-5f54p\""
Sep 07 03:55:56.815442 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:56.815421    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ztxnh"]
Sep 07 03:55:56.839248 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:56.839194    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:55:56.917737 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:56.917682    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/252adbfb-596a-4262-beb3-f761ecf8aa0c-data\") pod \"postgres-69fb4594cf-ztxnh\" (UID: \"252adbfb-596a-4262-beb3-f761ecf8aa0c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ztxnh"
Sep 07 03:55:56.917916 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:56.917772    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl5fk\" (UniqueName: \"kubernetes.io/projected/252adbfb-596a-4262-beb3-f761ecf8aa0c-kube-api-access-vl5fk\") pod \"postgres-69fb4594cf-ztxnh\" (UID: \"252adbfb-596a-4262-beb3-f761ecf8aa0c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ztxnh"
Sep 07 03:55:57.019092 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:57.019061    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/252adbfb-596a-4262-beb3-f761ecf8aa0c-data\") pod \"postgres-69fb4594cf-ztxnh\" (UID: \"252adbfb-596a-4262-beb3-f761ecf8aa0c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ztxnh"
Sep 07 03:55:57.019296 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:57.019111    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl5fk\" (UniqueName: \"kubernetes.io/projected/252adbfb-596a-4262-beb3-f761ecf8aa0c-kube-api-access-vl5fk\") pod \"postgres-69fb4594cf-ztxnh\" (UID: \"252adbfb-596a-4262-beb3-f761ecf8aa0c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ztxnh"
Sep 07 03:55:57.019521 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:57.019502    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/252adbfb-596a-4262-beb3-f761ecf8aa0c-data\") pod \"postgres-69fb4594cf-ztxnh\" (UID: \"252adbfb-596a-4262-beb3-f761ecf8aa0c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ztxnh"
Sep 07 03:55:57.028053 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:57.028033    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl5fk\" (UniqueName: \"kubernetes.io/projected/252adbfb-596a-4262-beb3-f761ecf8aa0c-kube-api-access-vl5fk\") pod \"postgres-69fb4594cf-ztxnh\" (UID: \"252adbfb-596a-4262-beb3-f761ecf8aa0c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ztxnh"
Sep 07 03:55:57.118545 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:57.118460    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ztxnh"
Sep 07 03:55:57.388662 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:57.388634    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ztxnh"]
Sep 07 03:55:57.390928 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:55:57.390893    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod252adbfb_596a_4262_beb3_f761ecf8aa0c.slice/crio-87ab1090e883d122189c3c56d618182a36b37798dc9035c88f2eeca2b9208907 WatchSource:0}: Error finding container 87ab1090e883d122189c3c56d618182a36b37798dc9035c88f2eeca2b9208907: Status 404 returned error can't find the container with id 87ab1090e883d122189c3c56d618182a36b37798dc9035c88f2eeca2b9208907
Sep 07 03:55:57.521353 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:57.521315    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3e87d2b-16fe-4cde-8d01-feea42bf7f94" path="/var/lib/kubelet/pods/f3e87d2b-16fe-4cde-8d01-feea42bf7f94/volumes"
Sep 07 03:55:57.571659 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:55:57.571625    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ztxnh" event={"ID":"252adbfb-596a-4262-beb3-f761ecf8aa0c","Type":"ContainerStarted","Data":"87ab1090e883d122189c3c56d618182a36b37798dc9035c88f2eeca2b9208907"}
Sep 07 03:56:02.599047 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:02.599010    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ztxnh" event={"ID":"252adbfb-596a-4262-beb3-f761ecf8aa0c","Type":"ContainerStarted","Data":"e975982e170d8d2a49aac52d409ab09f04a8d0d64707faf76d56a56d8d7fc1a5"}
Sep 07 03:56:02.599629 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:02.599116    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ztxnh"
Sep 07 03:56:02.615253 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:02.615185    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ztxnh" podStartSLOduration=1.867259722 podStartE2EDuration="6.61516918s" totalImagesPullingTime="4.747909458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:55:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:56:02.613455075 +0000 UTC m=+449.546935796" watchObservedRunningTime="2026-09-07 03:56:02.61516918 +0000 UTC m=+449.548649897"
Sep 07 03:56:08.636564 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:08.636536    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ztxnh"
Sep 07 03:56:08.966071 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:08.966031    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:56:09.376966 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:09.376879    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:56:11.317690 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:11.317651    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-bt56l"]
Sep 07 03:56:11.318125 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:11.317947    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-bt56l" podUID="e7e64784-85d0-408d-8d5f-5e9294e8a09c" containerName="authorino" containerID="cri-o://1e662722d880de7cdc2c477ac58a772e5ac6728317a03443b36c5153c7ab9437" gracePeriod=30
Sep 07 03:56:11.570548 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:11.570494    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-bt56l"
Sep 07 03:56:11.641183 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:11.641148    2583 generic.go:356] "Generic (PLEG): container finished" podID="e7e64784-85d0-408d-8d5f-5e9294e8a09c" containerID="1e662722d880de7cdc2c477ac58a772e5ac6728317a03443b36c5153c7ab9437" exitCode=0
Sep 07 03:56:11.641358 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:11.641192    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-bt56l"
Sep 07 03:56:11.641358 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:11.641235    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-bt56l" event={"ID":"e7e64784-85d0-408d-8d5f-5e9294e8a09c","Type":"ContainerDied","Data":"1e662722d880de7cdc2c477ac58a772e5ac6728317a03443b36c5153c7ab9437"}
Sep 07 03:56:11.641358 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:11.641267    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-bt56l" event={"ID":"e7e64784-85d0-408d-8d5f-5e9294e8a09c","Type":"ContainerDied","Data":"15214fcadaadd65f780d1158aa05fa2b5600dc02354aae37bbeb7a7e11997e9b"}
Sep 07 03:56:11.641358 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:11.641283    2583 scope.go:117] "RemoveContainer" containerID="1e662722d880de7cdc2c477ac58a772e5ac6728317a03443b36c5153c7ab9437"
Sep 07 03:56:11.649912 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:11.649889    2583 scope.go:117] "RemoveContainer" containerID="1e662722d880de7cdc2c477ac58a772e5ac6728317a03443b36c5153c7ab9437"
Sep 07 03:56:11.650196 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:56:11.650175    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e662722d880de7cdc2c477ac58a772e5ac6728317a03443b36c5153c7ab9437\": container with ID starting with 1e662722d880de7cdc2c477ac58a772e5ac6728317a03443b36c5153c7ab9437 not found: ID does not exist" containerID="1e662722d880de7cdc2c477ac58a772e5ac6728317a03443b36c5153c7ab9437"
Sep 07 03:56:11.650380 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:11.650317    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e662722d880de7cdc2c477ac58a772e5ac6728317a03443b36c5153c7ab9437"} err="failed to get container status \"1e662722d880de7cdc2c477ac58a772e5ac6728317a03443b36c5153c7ab9437\": rpc error: code = NotFound desc = could not find container \"1e662722d880de7cdc2c477ac58a772e5ac6728317a03443b36c5153c7ab9437\": container with ID starting with 1e662722d880de7cdc2c477ac58a772e5ac6728317a03443b36c5153c7ab9437 not found: ID does not exist"
Sep 07 03:56:11.653451 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:11.653433    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2nxp\" (UniqueName: \"kubernetes.io/projected/e7e64784-85d0-408d-8d5f-5e9294e8a09c-kube-api-access-c2nxp\") pod \"e7e64784-85d0-408d-8d5f-5e9294e8a09c\" (UID: \"e7e64784-85d0-408d-8d5f-5e9294e8a09c\") "
Sep 07 03:56:11.655559 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:11.655535    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7e64784-85d0-408d-8d5f-5e9294e8a09c-kube-api-access-c2nxp" (OuterVolumeSpecName: "kube-api-access-c2nxp") pod "e7e64784-85d0-408d-8d5f-5e9294e8a09c" (UID: "e7e64784-85d0-408d-8d5f-5e9294e8a09c"). InnerVolumeSpecName "kube-api-access-c2nxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 03:56:11.754807 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:11.754778    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2nxp\" (UniqueName: \"kubernetes.io/projected/e7e64784-85d0-408d-8d5f-5e9294e8a09c-kube-api-access-c2nxp\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:56:11.963078 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:11.963049    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-bt56l"]
Sep 07 03:56:11.967761 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:11.967737    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-bt56l"]
Sep 07 03:56:13.520749 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:13.520715    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7e64784-85d0-408d-8d5f-5e9294e8a09c" path="/var/lib/kubelet/pods/e7e64784-85d0-408d-8d5f-5e9294e8a09c/volumes"
Sep 07 03:56:16.669951 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.669915    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:56:16.685770 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.685731    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5d785576c9-j889w"]
Sep 07 03:56:16.686104 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.686091    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7e64784-85d0-408d-8d5f-5e9294e8a09c" containerName="authorino"
Sep 07 03:56:16.686157 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.686109    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7e64784-85d0-408d-8d5f-5e9294e8a09c" containerName="authorino"
Sep 07 03:56:16.686204 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.686194    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7e64784-85d0-408d-8d5f-5e9294e8a09c" containerName="authorino"
Sep 07 03:56:16.705909 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.705875    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d785576c9-j889w"]
Sep 07 03:56:16.706085 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.705999    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d785576c9-j889w"
Sep 07 03:56:16.708480 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.708460    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pl7hr\""
Sep 07 03:56:16.708624 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.708594    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 07 03:56:16.708686 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.708639    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 07 03:56:16.709763 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.709742    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 07 03:56:16.731238 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.731184    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:56:16.790376 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.790343    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:56:16.800058 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.800036    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57f950c5-460f-435e-94c8-07d127393ad8-metrics-certs\") pod \"maas-controller-5d785576c9-j889w\" (UID: \"57f950c5-460f-435e-94c8-07d127393ad8\") " pod="opendatahub/maas-controller-5d785576c9-j889w"
Sep 07 03:56:16.800131 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.800088    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wf85\" (UniqueName: \"kubernetes.io/projected/57f950c5-460f-435e-94c8-07d127393ad8-kube-api-access-2wf85\") pod \"maas-controller-5d785576c9-j889w\" (UID: \"57f950c5-460f-435e-94c8-07d127393ad8\") " pod="opendatahub/maas-controller-5d785576c9-j889w"
Sep 07 03:56:16.800196 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.800178    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/57f950c5-460f-435e-94c8-07d127393ad8-webhook-cert\") pod \"maas-controller-5d785576c9-j889w\" (UID: \"57f950c5-460f-435e-94c8-07d127393ad8\") " pod="opendatahub/maas-controller-5d785576c9-j889w"
Sep 07 03:56:16.856200 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.856159    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:56:16.859305 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.859277    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-9f656d559-wbsst"]
Sep 07 03:56:16.861708 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.861690    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9f656d559-wbsst"
Sep 07 03:56:16.871640 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.871618    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9f656d559-wbsst"]
Sep 07 03:56:16.901424 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.901395    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rhs6\" (UniqueName: \"kubernetes.io/projected/60352d1c-5ce1-4acd-a70c-befba153463a-kube-api-access-6rhs6\") pod \"maas-controller-9f656d559-wbsst\" (UID: \"60352d1c-5ce1-4acd-a70c-befba153463a\") " pod="opendatahub/maas-controller-9f656d559-wbsst"
Sep 07 03:56:16.901592 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.901439    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60352d1c-5ce1-4acd-a70c-befba153463a-webhook-cert\") pod \"maas-controller-9f656d559-wbsst\" (UID: \"60352d1c-5ce1-4acd-a70c-befba153463a\") " pod="opendatahub/maas-controller-9f656d559-wbsst"
Sep 07 03:56:16.901592 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.901455    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60352d1c-5ce1-4acd-a70c-befba153463a-metrics-certs\") pod \"maas-controller-9f656d559-wbsst\" (UID: \"60352d1c-5ce1-4acd-a70c-befba153463a\") " pod="opendatahub/maas-controller-9f656d559-wbsst"
Sep 07 03:56:16.901592 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.901538    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/57f950c5-460f-435e-94c8-07d127393ad8-webhook-cert\") pod \"maas-controller-5d785576c9-j889w\" (UID: \"57f950c5-460f-435e-94c8-07d127393ad8\") " pod="opendatahub/maas-controller-5d785576c9-j889w"
Sep 07 03:56:16.901592 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.901582    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57f950c5-460f-435e-94c8-07d127393ad8-metrics-certs\") pod \"maas-controller-5d785576c9-j889w\" (UID: \"57f950c5-460f-435e-94c8-07d127393ad8\") " pod="opendatahub/maas-controller-5d785576c9-j889w"
Sep 07 03:56:16.901783 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.901623    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wf85\" (UniqueName: \"kubernetes.io/projected/57f950c5-460f-435e-94c8-07d127393ad8-kube-api-access-2wf85\") pod \"maas-controller-5d785576c9-j889w\" (UID: \"57f950c5-460f-435e-94c8-07d127393ad8\") " pod="opendatahub/maas-controller-5d785576c9-j889w"
Sep 07 03:56:16.904073 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.904047    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/57f950c5-460f-435e-94c8-07d127393ad8-webhook-cert\") pod \"maas-controller-5d785576c9-j889w\" (UID: \"57f950c5-460f-435e-94c8-07d127393ad8\") " pod="opendatahub/maas-controller-5d785576c9-j889w"
Sep 07 03:56:16.904228 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.904191    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57f950c5-460f-435e-94c8-07d127393ad8-metrics-certs\") pod \"maas-controller-5d785576c9-j889w\" (UID: \"57f950c5-460f-435e-94c8-07d127393ad8\") " pod="opendatahub/maas-controller-5d785576c9-j889w"
Sep 07 03:56:16.910070 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:16.910047    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wf85\" (UniqueName: \"kubernetes.io/projected/57f950c5-460f-435e-94c8-07d127393ad8-kube-api-access-2wf85\") pod \"maas-controller-5d785576c9-j889w\" (UID: \"57f950c5-460f-435e-94c8-07d127393ad8\") " pod="opendatahub/maas-controller-5d785576c9-j889w"
Sep 07 03:56:17.002906 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:17.002874    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60352d1c-5ce1-4acd-a70c-befba153463a-webhook-cert\") pod \"maas-controller-9f656d559-wbsst\" (UID: \"60352d1c-5ce1-4acd-a70c-befba153463a\") " pod="opendatahub/maas-controller-9f656d559-wbsst"
Sep 07 03:56:17.002906 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:17.002904    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60352d1c-5ce1-4acd-a70c-befba153463a-metrics-certs\") pod \"maas-controller-9f656d559-wbsst\" (UID: \"60352d1c-5ce1-4acd-a70c-befba153463a\") " pod="opendatahub/maas-controller-9f656d559-wbsst"
Sep 07 03:56:17.003099 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:17.002981    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rhs6\" (UniqueName: \"kubernetes.io/projected/60352d1c-5ce1-4acd-a70c-befba153463a-kube-api-access-6rhs6\") pod \"maas-controller-9f656d559-wbsst\" (UID: \"60352d1c-5ce1-4acd-a70c-befba153463a\") " pod="opendatahub/maas-controller-9f656d559-wbsst"
Sep 07 03:56:17.005474 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:17.005445    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60352d1c-5ce1-4acd-a70c-befba153463a-metrics-certs\") pod \"maas-controller-9f656d559-wbsst\" (UID: \"60352d1c-5ce1-4acd-a70c-befba153463a\") " pod="opendatahub/maas-controller-9f656d559-wbsst"
Sep 07 03:56:17.005590 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:17.005515    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60352d1c-5ce1-4acd-a70c-befba153463a-webhook-cert\") pod \"maas-controller-9f656d559-wbsst\" (UID: \"60352d1c-5ce1-4acd-a70c-befba153463a\") " pod="opendatahub/maas-controller-9f656d559-wbsst"
Sep 07 03:56:17.015894 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:17.015871    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rhs6\" (UniqueName: \"kubernetes.io/projected/60352d1c-5ce1-4acd-a70c-befba153463a-kube-api-access-6rhs6\") pod \"maas-controller-9f656d559-wbsst\" (UID: \"60352d1c-5ce1-4acd-a70c-befba153463a\") " pod="opendatahub/maas-controller-9f656d559-wbsst"
Sep 07 03:56:17.016730 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:17.016711    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d785576c9-j889w"
Sep 07 03:56:17.173066 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:17.173034    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9f656d559-wbsst"
Sep 07 03:56:17.298076 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:17.298054    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d785576c9-j889w"]
Sep 07 03:56:17.299500 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:56:17.299468    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57f950c5_460f_435e_94c8_07d127393ad8.slice/crio-215a2deab18a8231c773e319cf4e8914a434e773d85b5585519bd061b6387d1d WatchSource:0}: Error finding container 215a2deab18a8231c773e319cf4e8914a434e773d85b5585519bd061b6387d1d: Status 404 returned error can't find the container with id 215a2deab18a8231c773e319cf4e8914a434e773d85b5585519bd061b6387d1d
Sep 07 03:56:17.445684 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:17.445658    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9f656d559-wbsst"]
Sep 07 03:56:17.447557 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:56:17.447530    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60352d1c_5ce1_4acd_a70c_befba153463a.slice/crio-9b847dd76d44952b37ea8a5a888962fe1b4ded3b9ca7b5f4a4100a99ee98e217 WatchSource:0}: Error finding container 9b847dd76d44952b37ea8a5a888962fe1b4ded3b9ca7b5f4a4100a99ee98e217: Status 404 returned error can't find the container with id 9b847dd76d44952b37ea8a5a888962fe1b4ded3b9ca7b5f4a4100a99ee98e217
Sep 07 03:56:17.670792 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:17.670701    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d785576c9-j889w" event={"ID":"57f950c5-460f-435e-94c8-07d127393ad8","Type":"ContainerStarted","Data":"215a2deab18a8231c773e319cf4e8914a434e773d85b5585519bd061b6387d1d"}
Sep 07 03:56:17.671925 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:17.671899    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9f656d559-wbsst" event={"ID":"60352d1c-5ce1-4acd-a70c-befba153463a","Type":"ContainerStarted","Data":"9b847dd76d44952b37ea8a5a888962fe1b4ded3b9ca7b5f4a4100a99ee98e217"}
Sep 07 03:56:20.087026 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:20.086969    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:56:20.186568 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:20.186531    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:56:20.689771 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:20.689736    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d785576c9-j889w" event={"ID":"57f950c5-460f-435e-94c8-07d127393ad8","Type":"ContainerStarted","Data":"1da91cec82cd98621423cf6b1eedce80e1d3d5df08b74ea60759db29d8d2fea4"}
Sep 07 03:56:20.689974 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:20.689929    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5d785576c9-j889w"
Sep 07 03:56:20.691377 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:20.691352    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9f656d559-wbsst" event={"ID":"60352d1c-5ce1-4acd-a70c-befba153463a","Type":"ContainerStarted","Data":"ca381ec4231fb99cceb81c4d7b32a13a658a8c5a77495d44bed5fef4f5706a7a"}
Sep 07 03:56:20.691563 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:20.691545    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-9f656d559-wbsst"
Sep 07 03:56:20.707581 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:20.707532    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5d785576c9-j889w" podStartSLOduration=2.118978171 podStartE2EDuration="4.707516139s" totalImagesPullingTime="2.588537968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:56:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:56:20.70653324 +0000 UTC m=+467.640013957" watchObservedRunningTime="2026-09-07 03:56:20.707516139 +0000 UTC m=+467.640996918"
Sep 07 03:56:20.724663 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:20.724608    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-9f656d559-wbsst" podStartSLOduration=2.2785238 podStartE2EDuration="4.724587146s" totalImagesPullingTime="2.446063346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:56:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:56:20.72352804 +0000 UTC m=+467.657008757" watchObservedRunningTime="2026-09-07 03:56:20.724587146 +0000 UTC m=+467.658067867"
Sep 07 03:56:31.698200 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:31.698171    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-9f656d559-wbsst"
Sep 07 03:56:31.698591 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:31.698270    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5d785576c9-j889w"
Sep 07 03:56:31.755911 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:31.755875    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5d785576c9-j889w"]
Sep 07 03:56:31.756137 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:31.756096    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5d785576c9-j889w" podUID="57f950c5-460f-435e-94c8-07d127393ad8" containerName="manager" containerID="cri-o://1da91cec82cd98621423cf6b1eedce80e1d3d5df08b74ea60759db29d8d2fea4" gracePeriod=30
Sep 07 03:56:32.002359 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:32.002337    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d785576c9-j889w"
Sep 07 03:56:32.139676 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:32.139641    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/57f950c5-460f-435e-94c8-07d127393ad8-webhook-cert\") pod \"57f950c5-460f-435e-94c8-07d127393ad8\" (UID: \"57f950c5-460f-435e-94c8-07d127393ad8\") "
Sep 07 03:56:32.139842 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:32.139723    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57f950c5-460f-435e-94c8-07d127393ad8-metrics-certs\") pod \"57f950c5-460f-435e-94c8-07d127393ad8\" (UID: \"57f950c5-460f-435e-94c8-07d127393ad8\") "
Sep 07 03:56:32.139842 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:32.139780    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wf85\" (UniqueName: \"kubernetes.io/projected/57f950c5-460f-435e-94c8-07d127393ad8-kube-api-access-2wf85\") pod \"57f950c5-460f-435e-94c8-07d127393ad8\" (UID: \"57f950c5-460f-435e-94c8-07d127393ad8\") "
Sep 07 03:56:32.142120 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:32.142085    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57f950c5-460f-435e-94c8-07d127393ad8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "57f950c5-460f-435e-94c8-07d127393ad8" (UID: "57f950c5-460f-435e-94c8-07d127393ad8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 03:56:32.142268 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:32.142129    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57f950c5-460f-435e-94c8-07d127393ad8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "57f950c5-460f-435e-94c8-07d127393ad8" (UID: "57f950c5-460f-435e-94c8-07d127393ad8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 03:56:32.142268 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:32.142155    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57f950c5-460f-435e-94c8-07d127393ad8-kube-api-access-2wf85" (OuterVolumeSpecName: "kube-api-access-2wf85") pod "57f950c5-460f-435e-94c8-07d127393ad8" (UID: "57f950c5-460f-435e-94c8-07d127393ad8"). InnerVolumeSpecName "kube-api-access-2wf85". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 03:56:32.241339 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:32.241257    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wf85\" (UniqueName: \"kubernetes.io/projected/57f950c5-460f-435e-94c8-07d127393ad8-kube-api-access-2wf85\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:56:32.241339 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:32.241283    2583 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/57f950c5-460f-435e-94c8-07d127393ad8-webhook-cert\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:56:32.241339 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:32.241292    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57f950c5-460f-435e-94c8-07d127393ad8-metrics-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 03:56:32.755858 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:32.755827    2583 generic.go:356] "Generic (PLEG): container finished" podID="57f950c5-460f-435e-94c8-07d127393ad8" containerID="1da91cec82cd98621423cf6b1eedce80e1d3d5df08b74ea60759db29d8d2fea4" exitCode=0
Sep 07 03:56:32.756245 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:32.755870    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d785576c9-j889w"
Sep 07 03:56:32.756245 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:32.755914    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d785576c9-j889w" event={"ID":"57f950c5-460f-435e-94c8-07d127393ad8","Type":"ContainerDied","Data":"1da91cec82cd98621423cf6b1eedce80e1d3d5df08b74ea60759db29d8d2fea4"}
Sep 07 03:56:32.756245 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:32.755967    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d785576c9-j889w" event={"ID":"57f950c5-460f-435e-94c8-07d127393ad8","Type":"ContainerDied","Data":"215a2deab18a8231c773e319cf4e8914a434e773d85b5585519bd061b6387d1d"}
Sep 07 03:56:32.756245 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:32.755995    2583 scope.go:117] "RemoveContainer" containerID="1da91cec82cd98621423cf6b1eedce80e1d3d5df08b74ea60759db29d8d2fea4"
Sep 07 03:56:32.765036 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:32.765018    2583 scope.go:117] "RemoveContainer" containerID="1da91cec82cd98621423cf6b1eedce80e1d3d5df08b74ea60759db29d8d2fea4"
Sep 07 03:56:32.765358 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:56:32.765335    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1da91cec82cd98621423cf6b1eedce80e1d3d5df08b74ea60759db29d8d2fea4\": container with ID starting with 1da91cec82cd98621423cf6b1eedce80e1d3d5df08b74ea60759db29d8d2fea4 not found: ID does not exist" containerID="1da91cec82cd98621423cf6b1eedce80e1d3d5df08b74ea60759db29d8d2fea4"
Sep 07 03:56:32.765419 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:32.765370    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1da91cec82cd98621423cf6b1eedce80e1d3d5df08b74ea60759db29d8d2fea4"} err="failed to get container status \"1da91cec82cd98621423cf6b1eedce80e1d3d5df08b74ea60759db29d8d2fea4\": rpc error: code = NotFound desc = could not find container \"1da91cec82cd98621423cf6b1eedce80e1d3d5df08b74ea60759db29d8d2fea4\": container with ID starting with 1da91cec82cd98621423cf6b1eedce80e1d3d5df08b74ea60759db29d8d2fea4 not found: ID does not exist"
Sep 07 03:56:32.777893 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:32.777873    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5d785576c9-j889w"]
Sep 07 03:56:32.781737 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:32.781714    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5d785576c9-j889w"]
Sep 07 03:56:33.521365 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:33.521335    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57f950c5-460f-435e-94c8-07d127393ad8" path="/var/lib/kubelet/pods/57f950c5-460f-435e-94c8-07d127393ad8/volumes"
Sep 07 03:56:34.588462 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:34.588425    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:56:34.644715 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:34.644683    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:56:35.963881 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:35.963843    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:56:51.936976 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:51.936936    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:56:52.034020 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:52.033991    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:56:53.320418 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.320379    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:56:53.438967 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.438932    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:56:53.487679 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.487643    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"]
Sep 07 03:56:53.488005 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.487992    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57f950c5-460f-435e-94c8-07d127393ad8" containerName="manager"
Sep 07 03:56:53.488056 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.488007    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="57f950c5-460f-435e-94c8-07d127393ad8" containerName="manager"
Sep 07 03:56:53.488098 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.488062    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="57f950c5-460f-435e-94c8-07d127393ad8" containerName="manager"
Sep 07 03:56:53.493847 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.493828    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"
Sep 07 03:56:53.498262 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.498191    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 07 03:56:53.504075 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.504054    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-m9mvz\""
Sep 07 03:56:53.518931 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.518898    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 07 03:56:53.522173 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.522147    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"]
Sep 07 03:56:53.527623 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.527585    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-metrics-certs\") pod \"maas-api-86d6f5487f-wgcpv\" (UID: \"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2\") " pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"
Sep 07 03:56:53.527623 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.527617    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-maas-api-tls\") pod \"maas-api-86d6f5487f-wgcpv\" (UID: \"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2\") " pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"
Sep 07 03:56:53.527816 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.527702    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp7qx\" (UniqueName: \"kubernetes.io/projected/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-kube-api-access-kp7qx\") pod \"maas-api-86d6f5487f-wgcpv\" (UID: \"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2\") " pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"
Sep 07 03:56:53.527816 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.527737    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-openshift-service-ca\") pod \"maas-api-86d6f5487f-wgcpv\" (UID: \"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2\") " pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"
Sep 07 03:56:53.628718 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.628624    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp7qx\" (UniqueName: \"kubernetes.io/projected/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-kube-api-access-kp7qx\") pod \"maas-api-86d6f5487f-wgcpv\" (UID: \"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2\") " pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"
Sep 07 03:56:53.628718 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.628675    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-openshift-service-ca\") pod \"maas-api-86d6f5487f-wgcpv\" (UID: \"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2\") " pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"
Sep 07 03:56:53.628914 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.628759    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-metrics-certs\") pod \"maas-api-86d6f5487f-wgcpv\" (UID: \"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2\") " pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"
Sep 07 03:56:53.628914 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.628790    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-maas-api-tls\") pod \"maas-api-86d6f5487f-wgcpv\" (UID: \"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2\") " pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"
Sep 07 03:56:53.629012 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:56:53.628947    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 07 03:56:53.629057 ip-10-0-132-175 kubenswrapper[2583]: E0907 03:56:53.629014    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-maas-api-tls podName:9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2 nodeName:}" failed. No retries permitted until 2026-09-07 03:56:54.128993841 +0000 UTC m=+501.062474539 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-maas-api-tls") pod "maas-api-86d6f5487f-wgcpv" (UID: "9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2") : secret "maas-api-serving-cert" not found
Sep 07 03:56:53.629514 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.629449    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-openshift-service-ca\") pod \"maas-api-86d6f5487f-wgcpv\" (UID: \"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2\") " pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"
Sep 07 03:56:53.631226 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.631194    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-metrics-certs\") pod \"maas-api-86d6f5487f-wgcpv\" (UID: \"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2\") " pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"
Sep 07 03:56:53.641917 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.641895    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp7qx\" (UniqueName: \"kubernetes.io/projected/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-kube-api-access-kp7qx\") pod \"maas-api-86d6f5487f-wgcpv\" (UID: \"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2\") " pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"
Sep 07 03:56:53.654242 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.654202    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:56:53.761439 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.761402    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:56:53.880380 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.880292    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:56:53.980078 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:53.980038    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:56:54.074022 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:54.073987    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-5m597"]
Sep 07 03:56:54.077374 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:54.077355    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5m597"
Sep 07 03:56:54.080470 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:54.080444    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-hwdzq\""
Sep 07 03:56:54.080586 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:54.080485    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 07 03:56:54.088272 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:54.088249    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-5m597"]
Sep 07 03:56:54.132661 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:54.132589    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-maas-api-tls\") pod \"maas-api-86d6f5487f-wgcpv\" (UID: \"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2\") " pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"
Sep 07 03:56:54.132661 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:54.132645    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ffd0e95-1557-4337-b585-7cd619dbc579-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-5m597\" (UID: \"9ffd0e95-1557-4337-b585-7cd619dbc579\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5m597"
Sep 07 03:56:54.132854 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:54.132686    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z6j8\" (UniqueName: \"kubernetes.io/projected/9ffd0e95-1557-4337-b585-7cd619dbc579-kube-api-access-4z6j8\") pod \"payload-pre-processing-59fbc47bf8-5m597\" (UID: \"9ffd0e95-1557-4337-b585-7cd619dbc579\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5m597"
Sep 07 03:56:54.135095 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:54.135070    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-maas-api-tls\") pod \"maas-api-86d6f5487f-wgcpv\" (UID: \"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2\") " pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"
Sep 07 03:56:54.233592 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:54.233544    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z6j8\" (UniqueName: \"kubernetes.io/projected/9ffd0e95-1557-4337-b585-7cd619dbc579-kube-api-access-4z6j8\") pod \"payload-pre-processing-59fbc47bf8-5m597\" (UID: \"9ffd0e95-1557-4337-b585-7cd619dbc579\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5m597"
Sep 07 03:56:54.233793 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:54.233650    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ffd0e95-1557-4337-b585-7cd619dbc579-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-5m597\" (UID: \"9ffd0e95-1557-4337-b585-7cd619dbc579\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5m597"
Sep 07 03:56:54.234376 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:54.234352    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ffd0e95-1557-4337-b585-7cd619dbc579-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-5m597\" (UID: \"9ffd0e95-1557-4337-b585-7cd619dbc579\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5m597"
Sep 07 03:56:54.241465 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:54.241442    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z6j8\" (UniqueName: \"kubernetes.io/projected/9ffd0e95-1557-4337-b585-7cd619dbc579-kube-api-access-4z6j8\") pod \"payload-pre-processing-59fbc47bf8-5m597\" (UID: \"9ffd0e95-1557-4337-b585-7cd619dbc579\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5m597"
Sep 07 03:56:54.389183 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:54.389080    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5m597"
Sep 07 03:56:54.404169 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:54.404126    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"
Sep 07 03:56:54.668689 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:54.668663    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-5m597"]
Sep 07 03:56:54.671288 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:56:54.671259    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ffd0e95_1557_4337_b585_7cd619dbc579.slice/crio-cc4dfcc885534cd2bf043c1d3ecd2c6c9d30bebb6cb22091aec1c51ddc0111f5 WatchSource:0}: Error finding container cc4dfcc885534cd2bf043c1d3ecd2c6c9d30bebb6cb22091aec1c51ddc0111f5: Status 404 returned error can't find the container with id cc4dfcc885534cd2bf043c1d3ecd2c6c9d30bebb6cb22091aec1c51ddc0111f5
Sep 07 03:56:54.695890 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:54.695845    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"]
Sep 07 03:56:54.698359 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:56:54.698330    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e8cb9d1_b0ac_4f9e_82c3_06b790c2e0f2.slice/crio-415009680b3d405143d193d8ea6994d2332512448390175eee1ebda32cb7abc9 WatchSource:0}: Error finding container 415009680b3d405143d193d8ea6994d2332512448390175eee1ebda32cb7abc9: Status 404 returned error can't find the container with id 415009680b3d405143d193d8ea6994d2332512448390175eee1ebda32cb7abc9
Sep 07 03:56:54.854275 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:54.854236    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5m597" event={"ID":"9ffd0e95-1557-4337-b585-7cd619dbc579","Type":"ContainerStarted","Data":"cc4dfcc885534cd2bf043c1d3ecd2c6c9d30bebb6cb22091aec1c51ddc0111f5"}
Sep 07 03:56:54.855463 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:54.855437    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv" event={"ID":"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2","Type":"ContainerStarted","Data":"415009680b3d405143d193d8ea6994d2332512448390175eee1ebda32cb7abc9"}
Sep 07 03:56:56.867921 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:56.867834    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5m597" event={"ID":"9ffd0e95-1557-4337-b585-7cd619dbc579","Type":"ContainerStarted","Data":"d003da550d9b0ca0f077ae6998752462dacae7c8733f87eb89a2347589d8d59b"}
Sep 07 03:56:56.868400 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:56.867928    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5m597"
Sep 07 03:56:56.869520 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:56.869499    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv" event={"ID":"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2","Type":"ContainerStarted","Data":"3bb3cdbacbb13a89327626aa50ffce1d4d2f87524a1406c497f7392419e7486a"}
Sep 07 03:56:56.869646 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:56.869621    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"
Sep 07 03:56:56.884613 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:56.884567    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5m597" podStartSLOduration=1.030767331 podStartE2EDuration="2.884546064s" totalImagesPullingTime="1.853778733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:56:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:56:56.883246733 +0000 UTC m=+503.816727451" watchObservedRunningTime="2026-09-07 03:56:56.884546064 +0000 UTC m=+503.818026779"
Sep 07 03:56:56.900300 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:56:56.900246    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv" podStartSLOduration=2.079147219 podStartE2EDuration="3.900228959s" totalImagesPullingTime="1.82108174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:56:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:56:56.898962128 +0000 UTC m=+503.832442848" watchObservedRunningTime="2026-09-07 03:56:56.900228959 +0000 UTC m=+503.833709667"
Sep 07 03:57:02.879659 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:02.879625    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"
Sep 07 03:57:03.883757 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:03.883725    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:57:03.987399 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:03.987355    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:57:07.876110 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:07.876081    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-5m597"
Sep 07 03:57:39.507532 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:39.507455    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:57:53.474992 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.474958    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"]
Sep 07 03:57:53.480292 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.480271    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:57:53.483012 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.482986    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 07 03:57:53.483156 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.483101    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 07 03:57:53.484099 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.484080    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 07 03:57:53.484248 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.484103    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-v92z7\""
Sep 07 03:57:53.489416 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.489393    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"]
Sep 07 03:57:53.501882 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.501853    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:57:53.549891 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.549856    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx9db\" (UniqueName: \"kubernetes.io/projected/dcf567d8-c91d-4855-8494-77316fc195be-kube-api-access-hx9db\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9td7c\" (UID: \"dcf567d8-c91d-4855-8494-77316fc195be\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:57:53.550062 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.549915    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcf567d8-c91d-4855-8494-77316fc195be-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9td7c\" (UID: \"dcf567d8-c91d-4855-8494-77316fc195be\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:57:53.550062 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.549946    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcf567d8-c91d-4855-8494-77316fc195be-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9td7c\" (UID: \"dcf567d8-c91d-4855-8494-77316fc195be\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:57:53.550062 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.549968    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcf567d8-c91d-4855-8494-77316fc195be-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9td7c\" (UID: \"dcf567d8-c91d-4855-8494-77316fc195be\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:57:53.550062 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.549981    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcf567d8-c91d-4855-8494-77316fc195be-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9td7c\" (UID: \"dcf567d8-c91d-4855-8494-77316fc195be\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:57:53.550062 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.550009    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcf567d8-c91d-4855-8494-77316fc195be-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9td7c\" (UID: \"dcf567d8-c91d-4855-8494-77316fc195be\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:57:53.651171 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.651142    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx9db\" (UniqueName: \"kubernetes.io/projected/dcf567d8-c91d-4855-8494-77316fc195be-kube-api-access-hx9db\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9td7c\" (UID: \"dcf567d8-c91d-4855-8494-77316fc195be\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:57:53.651405 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.651198    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcf567d8-c91d-4855-8494-77316fc195be-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9td7c\" (UID: \"dcf567d8-c91d-4855-8494-77316fc195be\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:57:53.651405 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.651237    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcf567d8-c91d-4855-8494-77316fc195be-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9td7c\" (UID: \"dcf567d8-c91d-4855-8494-77316fc195be\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:57:53.651405 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.651263    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcf567d8-c91d-4855-8494-77316fc195be-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9td7c\" (UID: \"dcf567d8-c91d-4855-8494-77316fc195be\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:57:53.651569 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.651402    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcf567d8-c91d-4855-8494-77316fc195be-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9td7c\" (UID: \"dcf567d8-c91d-4855-8494-77316fc195be\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:57:53.651569 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.651461    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcf567d8-c91d-4855-8494-77316fc195be-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9td7c\" (UID: \"dcf567d8-c91d-4855-8494-77316fc195be\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:57:53.651818 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.651793    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dcf567d8-c91d-4855-8494-77316fc195be-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9td7c\" (UID: \"dcf567d8-c91d-4855-8494-77316fc195be\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:57:53.651904 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.651820    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dcf567d8-c91d-4855-8494-77316fc195be-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9td7c\" (UID: \"dcf567d8-c91d-4855-8494-77316fc195be\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:57:53.652321 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.652294    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dcf567d8-c91d-4855-8494-77316fc195be-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9td7c\" (UID: \"dcf567d8-c91d-4855-8494-77316fc195be\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:57:53.654004 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.653984    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcf567d8-c91d-4855-8494-77316fc195be-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9td7c\" (UID: \"dcf567d8-c91d-4855-8494-77316fc195be\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:57:53.654114 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.654101    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dcf567d8-c91d-4855-8494-77316fc195be-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9td7c\" (UID: \"dcf567d8-c91d-4855-8494-77316fc195be\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:57:53.660628 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.660595    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx9db\" (UniqueName: \"kubernetes.io/projected/dcf567d8-c91d-4855-8494-77316fc195be-kube-api-access-hx9db\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9td7c\" (UID: \"dcf567d8-c91d-4855-8494-77316fc195be\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:57:53.791275 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:53.791169    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:57:54.066651 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:54.066581    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"]
Sep 07 03:57:54.069736 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:57:54.069705    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcf567d8_c91d_4855_8494_77316fc195be.slice/crio-e009e36cd4ea6b9974247b26b32587a90e4a53eb68877793a2e6494745ac3d6c WatchSource:0}: Error finding container e009e36cd4ea6b9974247b26b32587a90e4a53eb68877793a2e6494745ac3d6c: Status 404 returned error can't find the container with id e009e36cd4ea6b9974247b26b32587a90e4a53eb68877793a2e6494745ac3d6c
Sep 07 03:57:54.126454 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:54.126415    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c" event={"ID":"dcf567d8-c91d-4855-8494-77316fc195be","Type":"ContainerStarted","Data":"e009e36cd4ea6b9974247b26b32587a90e4a53eb68877793a2e6494745ac3d6c"}
Sep 07 03:57:56.136943 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:56.136908    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c" event={"ID":"dcf567d8-c91d-4855-8494-77316fc195be","Type":"ContainerStarted","Data":"ed150b61dd710b86f860ec5a1794b6eaf3cc2c6fe56cb191ef9f998a55a5dbfa"}
Sep 07 03:57:56.156494 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:57:56.156448    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c" podStartSLOduration=2.136386804 podStartE2EDuration="3.156431982s" totalImagesPullingTime="1.020045178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:57:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:57:56.155247054 +0000 UTC m=+563.088727772" watchObservedRunningTime="2026-09-07 03:57:56.156431982 +0000 UTC m=+563.089912699"
Sep 07 03:58:00.098515 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:00.098479    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:58:03.791880 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:03.791834    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:58:03.792317 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:03.791897    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:58:03.804521 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:03.804438    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:58:04.186197 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:04.186172    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9td7c"
Sep 07 03:58:07.608355 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:07.608320    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:58:14.411867 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:14.411835    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:58:23.172803 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.172761    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"]
Sep 07 03:58:23.177496 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.177471    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:23.180090 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.180064    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 07 03:58:23.188986 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.188959    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"]
Sep 07 03:58:23.211649 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.211616    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:58:23.316981 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.316947    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b52a8bd4-3572-41fc-95ed-cd4d884023b6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cpsc4\" (UID: \"b52a8bd4-3572-41fc-95ed-cd4d884023b6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:23.317163 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.316992    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b52a8bd4-3572-41fc-95ed-cd4d884023b6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cpsc4\" (UID: \"b52a8bd4-3572-41fc-95ed-cd4d884023b6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:23.317163 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.317078    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b52a8bd4-3572-41fc-95ed-cd4d884023b6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cpsc4\" (UID: \"b52a8bd4-3572-41fc-95ed-cd4d884023b6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:23.317163 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.317103    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmq5g\" (UniqueName: \"kubernetes.io/projected/b52a8bd4-3572-41fc-95ed-cd4d884023b6-kube-api-access-xmq5g\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cpsc4\" (UID: \"b52a8bd4-3572-41fc-95ed-cd4d884023b6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:23.317163 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.317133    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b52a8bd4-3572-41fc-95ed-cd4d884023b6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cpsc4\" (UID: \"b52a8bd4-3572-41fc-95ed-cd4d884023b6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:23.317351 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.317178    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b52a8bd4-3572-41fc-95ed-cd4d884023b6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cpsc4\" (UID: \"b52a8bd4-3572-41fc-95ed-cd4d884023b6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:23.418327 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.418291    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b52a8bd4-3572-41fc-95ed-cd4d884023b6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cpsc4\" (UID: \"b52a8bd4-3572-41fc-95ed-cd4d884023b6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:23.418327 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.418328    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmq5g\" (UniqueName: \"kubernetes.io/projected/b52a8bd4-3572-41fc-95ed-cd4d884023b6-kube-api-access-xmq5g\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cpsc4\" (UID: \"b52a8bd4-3572-41fc-95ed-cd4d884023b6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:23.418541 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.418444    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b52a8bd4-3572-41fc-95ed-cd4d884023b6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cpsc4\" (UID: \"b52a8bd4-3572-41fc-95ed-cd4d884023b6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:23.418541 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.418486    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b52a8bd4-3572-41fc-95ed-cd4d884023b6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cpsc4\" (UID: \"b52a8bd4-3572-41fc-95ed-cd4d884023b6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:23.418541 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.418530    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b52a8bd4-3572-41fc-95ed-cd4d884023b6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cpsc4\" (UID: \"b52a8bd4-3572-41fc-95ed-cd4d884023b6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:23.418664 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.418565    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b52a8bd4-3572-41fc-95ed-cd4d884023b6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cpsc4\" (UID: \"b52a8bd4-3572-41fc-95ed-cd4d884023b6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:23.418737 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.418714    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b52a8bd4-3572-41fc-95ed-cd4d884023b6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cpsc4\" (UID: \"b52a8bd4-3572-41fc-95ed-cd4d884023b6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:23.418958 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.418931    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b52a8bd4-3572-41fc-95ed-cd4d884023b6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cpsc4\" (UID: \"b52a8bd4-3572-41fc-95ed-cd4d884023b6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:23.419080 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.418941    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b52a8bd4-3572-41fc-95ed-cd4d884023b6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cpsc4\" (UID: \"b52a8bd4-3572-41fc-95ed-cd4d884023b6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:23.420863 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.420838    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b52a8bd4-3572-41fc-95ed-cd4d884023b6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cpsc4\" (UID: \"b52a8bd4-3572-41fc-95ed-cd4d884023b6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:23.421267 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.421245    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b52a8bd4-3572-41fc-95ed-cd4d884023b6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cpsc4\" (UID: \"b52a8bd4-3572-41fc-95ed-cd4d884023b6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:23.427060 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.427005    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmq5g\" (UniqueName: \"kubernetes.io/projected/b52a8bd4-3572-41fc-95ed-cd4d884023b6-kube-api-access-xmq5g\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cpsc4\" (UID: \"b52a8bd4-3572-41fc-95ed-cd4d884023b6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:23.488948 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.488913    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:23.760855 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:23.760824    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"]
Sep 07 03:58:23.764023 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:58:23.763994    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb52a8bd4_3572_41fc_95ed_cd4d884023b6.slice/crio-04d2d7e05c5461745535054aa2f358a69c3f67f9dd780a109e192e5907854ac0 WatchSource:0}: Error finding container 04d2d7e05c5461745535054aa2f358a69c3f67f9dd780a109e192e5907854ac0: Status 404 returned error can't find the container with id 04d2d7e05c5461745535054aa2f358a69c3f67f9dd780a109e192e5907854ac0
Sep 07 03:58:24.265968 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:24.265930    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4" event={"ID":"b52a8bd4-3572-41fc-95ed-cd4d884023b6","Type":"ContainerStarted","Data":"044e91bf71d527fed42c1897c53ae058976d773bf436bf15295c7e3a55b99c31"}
Sep 07 03:58:24.266436 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:24.265989    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4" event={"ID":"b52a8bd4-3572-41fc-95ed-cd4d884023b6","Type":"ContainerStarted","Data":"04d2d7e05c5461745535054aa2f358a69c3f67f9dd780a109e192e5907854ac0"}
Sep 07 03:58:24.283198 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:24.283139    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4" podStartSLOduration=1.100712412 podStartE2EDuration="1.283122555s" totalImagesPullingTime="182.410143ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:58:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:58:24.282759808 +0000 UTC m=+591.216240537" watchObservedRunningTime="2026-09-07 03:58:24.283122555 +0000 UTC m=+591.216603272"
Sep 07 03:58:33.489295 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:33.489259    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:33.489706 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:33.489314    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:33.501640 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:33.501615    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:33.642472 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:33.642444    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:58:34.329939 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:34.329909    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cpsc4"
Sep 07 03:58:36.672585 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.672554    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"]
Sep 07 03:58:36.676232 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.676190    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:58:36.678730 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.678707    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 07 03:58:36.686295 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.686271    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"]
Sep 07 03:58:36.702918 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.702886    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:58:36.732847 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.732815    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05856c4d-524e-4235-b0aa-a5e0ecac3c23-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf\" (UID: \"05856c4d-524e-4235-b0aa-a5e0ecac3c23\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:58:36.732991 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.732853    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05856c4d-524e-4235-b0aa-a5e0ecac3c23-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf\" (UID: \"05856c4d-524e-4235-b0aa-a5e0ecac3c23\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:58:36.732991 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.732932    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05856c4d-524e-4235-b0aa-a5e0ecac3c23-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf\" (UID: \"05856c4d-524e-4235-b0aa-a5e0ecac3c23\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:58:36.732991 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.732964    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05856c4d-524e-4235-b0aa-a5e0ecac3c23-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf\" (UID: \"05856c4d-524e-4235-b0aa-a5e0ecac3c23\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:58:36.732991 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.732988    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05856c4d-524e-4235-b0aa-a5e0ecac3c23-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf\" (UID: \"05856c4d-524e-4235-b0aa-a5e0ecac3c23\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:58:36.733123 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.733029    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slgw9\" (UniqueName: \"kubernetes.io/projected/05856c4d-524e-4235-b0aa-a5e0ecac3c23-kube-api-access-slgw9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf\" (UID: \"05856c4d-524e-4235-b0aa-a5e0ecac3c23\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:58:36.834194 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.834162    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05856c4d-524e-4235-b0aa-a5e0ecac3c23-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf\" (UID: \"05856c4d-524e-4235-b0aa-a5e0ecac3c23\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:58:36.834384 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.834199    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05856c4d-524e-4235-b0aa-a5e0ecac3c23-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf\" (UID: \"05856c4d-524e-4235-b0aa-a5e0ecac3c23\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:58:36.834384 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.834282    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05856c4d-524e-4235-b0aa-a5e0ecac3c23-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf\" (UID: \"05856c4d-524e-4235-b0aa-a5e0ecac3c23\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:58:36.834384 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.834309    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05856c4d-524e-4235-b0aa-a5e0ecac3c23-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf\" (UID: \"05856c4d-524e-4235-b0aa-a5e0ecac3c23\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:58:36.834538 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.834438    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05856c4d-524e-4235-b0aa-a5e0ecac3c23-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf\" (UID: \"05856c4d-524e-4235-b0aa-a5e0ecac3c23\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:58:36.834590 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.834535    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slgw9\" (UniqueName: \"kubernetes.io/projected/05856c4d-524e-4235-b0aa-a5e0ecac3c23-kube-api-access-slgw9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf\" (UID: \"05856c4d-524e-4235-b0aa-a5e0ecac3c23\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:58:36.834771 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.834746    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05856c4d-524e-4235-b0aa-a5e0ecac3c23-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf\" (UID: \"05856c4d-524e-4235-b0aa-a5e0ecac3c23\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:58:36.834897 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.834777    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05856c4d-524e-4235-b0aa-a5e0ecac3c23-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf\" (UID: \"05856c4d-524e-4235-b0aa-a5e0ecac3c23\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:58:36.834897 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.834746    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05856c4d-524e-4235-b0aa-a5e0ecac3c23-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf\" (UID: \"05856c4d-524e-4235-b0aa-a5e0ecac3c23\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:58:36.837167 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.837141    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05856c4d-524e-4235-b0aa-a5e0ecac3c23-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf\" (UID: \"05856c4d-524e-4235-b0aa-a5e0ecac3c23\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:58:36.837315 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.837300    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05856c4d-524e-4235-b0aa-a5e0ecac3c23-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf\" (UID: \"05856c4d-524e-4235-b0aa-a5e0ecac3c23\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:58:36.842666 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.842641    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slgw9\" (UniqueName: \"kubernetes.io/projected/05856c4d-524e-4235-b0aa-a5e0ecac3c23-kube-api-access-slgw9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf\" (UID: \"05856c4d-524e-4235-b0aa-a5e0ecac3c23\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:58:36.985987 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:36.985951    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:58:37.259324 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:37.259277    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"]
Sep 07 03:58:37.261568 ip-10-0-132-175 kubenswrapper[2583]: W0907 03:58:37.261543    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05856c4d_524e_4235_b0aa_a5e0ecac3c23.slice/crio-cd304aaf5a04d2d6af3c51687b25a3a07811c6abe0d418fc12be2dc12915c808 WatchSource:0}: Error finding container cd304aaf5a04d2d6af3c51687b25a3a07811c6abe0d418fc12be2dc12915c808: Status 404 returned error can't find the container with id cd304aaf5a04d2d6af3c51687b25a3a07811c6abe0d418fc12be2dc12915c808
Sep 07 03:58:37.330286 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:37.330249    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf" event={"ID":"05856c4d-524e-4235-b0aa-a5e0ecac3c23","Type":"ContainerStarted","Data":"cd304aaf5a04d2d6af3c51687b25a3a07811c6abe0d418fc12be2dc12915c808"}
Sep 07 03:58:38.336655 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:38.336621    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf" event={"ID":"05856c4d-524e-4235-b0aa-a5e0ecac3c23","Type":"ContainerStarted","Data":"cf68d3e94bb7008798f4351952bc93b28be985911ae58d4df9de65344f0f4e54"}
Sep 07 03:58:38.353998 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:38.353945    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf" podStartSLOduration=2.194699133 podStartE2EDuration="2.35392803s" totalImagesPullingTime="159.228897ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 03:58:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 03:58:38.353235554 +0000 UTC m=+605.286716273" watchObservedRunningTime="2026-09-07 03:58:38.35392803 +0000 UTC m=+605.287408748"
Sep 07 03:58:46.986803 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:46.986756    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:58:46.987394 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:46.986821    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:58:46.999664 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:46.999638    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:58:47.392591 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:58:47.392507    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf"
Sep 07 03:59:24.502314 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:59:24.502277    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:59:30.739589 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:59:30.739560    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:59:30.838488 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:59:30.838456    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 03:59:59.947369 ip-10-0-132-175 kubenswrapper[2583]: I0907 03:59:59.947328    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:00.049008 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:00.048975    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:01.039685 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:01.039646    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:01.185000 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:01.184965    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:16.330883 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:16.330817    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:16.437804 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:16.437769    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:16.652037 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:16.651960    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:16.746584 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:16.746553    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:17.730430 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:17.730353    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:17.830548 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:17.830515    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:18.126292 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:18.126088    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:19.753740 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:19.753705    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:19.853618 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:19.853582    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:20.989200 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:20.989102    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"]
Sep 07 04:00:20.992909 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:20.992887    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"
Sep 07 04:00:20.995364 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:20.995342    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-1038cd-serving-cert\""
Sep 07 04:00:21.003300 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:21.003270    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"]
Sep 07 04:00:21.089074 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:21.089036    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:21.101329 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:21.101295    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt\" (UID: \"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"
Sep 07 04:00:21.101522 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:21.101357    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-metrics-certs\") pod \"maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt\" (UID: \"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"
Sep 07 04:00:21.101522 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:21.101470    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9bhb\" (UniqueName: \"kubernetes.io/projected/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-kube-api-access-p9bhb\") pod \"maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt\" (UID: \"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"
Sep 07 04:00:21.101522 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:21.101502    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-maas-api-tls\") pod \"maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt\" (UID: \"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"
Sep 07 04:00:21.189592 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:21.189556    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:21.202791 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:21.202757    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt\" (UID: \"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"
Sep 07 04:00:21.202896 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:21.202815    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-metrics-certs\") pod \"maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt\" (UID: \"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"
Sep 07 04:00:21.202896 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:21.202866    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9bhb\" (UniqueName: \"kubernetes.io/projected/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-kube-api-access-p9bhb\") pod \"maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt\" (UID: \"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"
Sep 07 04:00:21.202896 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:21.202883    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-maas-api-tls\") pod \"maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt\" (UID: \"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"
Sep 07 04:00:21.203110 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:00:21.203032    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-serving-cert: secret "maas-api-e2e-inf-a-1038cd-serving-cert" not found
Sep 07 04:00:21.203165 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:00:21.203137    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-maas-api-tls podName:f9b44b17-73cc-4c87-8d64-8bcf7bb10f24 nodeName:}" failed. No retries permitted until 2026-09-07 04:00:21.703112391 +0000 UTC m=+708.636593087 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-maas-api-tls") pod "maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt" (UID: "f9b44b17-73cc-4c87-8d64-8bcf7bb10f24") : secret "maas-api-e2e-inf-a-1038cd-serving-cert" not found
Sep 07 04:00:21.203505 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:21.203482    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt\" (UID: \"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"
Sep 07 04:00:21.205588 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:21.205569    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-metrics-certs\") pod \"maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt\" (UID: \"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"
Sep 07 04:00:21.211547 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:21.211517    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9bhb\" (UniqueName: \"kubernetes.io/projected/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-kube-api-access-p9bhb\") pod \"maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt\" (UID: \"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"
Sep 07 04:00:21.543925 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:21.543890    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:21.646477 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:21.646447    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:21.708462 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:21.708430    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-maas-api-tls\") pod \"maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt\" (UID: \"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"
Sep 07 04:00:21.711093 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:21.711014    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-maas-api-tls\") pod \"maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt\" (UID: \"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"
Sep 07 04:00:21.720040 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:21.720006    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:21.905090 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:21.905000    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"
Sep 07 04:00:22.186436 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:22.186411    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"]
Sep 07 04:00:22.190819 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:00:22.190772    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9b44b17_73cc_4c87_8d64_8bcf7bb10f24.slice/crio-09e36c437ff0d5ff487bb9ebcf349d675440f0a6fc4920840e854f78947de592 WatchSource:0}: Error finding container 09e36c437ff0d5ff487bb9ebcf349d675440f0a6fc4920840e854f78947de592: Status 404 returned error can't find the container with id 09e36c437ff0d5ff487bb9ebcf349d675440f0a6fc4920840e854f78947de592
Sep 07 04:00:22.192616 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:22.192597    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 04:00:22.845428 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:22.845391    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt" event={"ID":"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24","Type":"ContainerStarted","Data":"a305ea081848210b855c8f0fb2329ddc73e96f1ae61598a5e20d5743d5df0fb5"}
Sep 07 04:00:22.845626 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:22.845481    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt" event={"ID":"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24","Type":"ContainerStarted","Data":"09e36c437ff0d5ff487bb9ebcf349d675440f0a6fc4920840e854f78947de592"}
Sep 07 04:00:22.845626 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:22.845510    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"
Sep 07 04:00:22.864959 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:22.864903    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt" podStartSLOduration=2.393446273 podStartE2EDuration="2.864885329s" totalImagesPullingTime="471.439056ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:00:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:00:22.864783985 +0000 UTC m=+709.798264703" watchObservedRunningTime="2026-09-07 04:00:22.864885329 +0000 UTC m=+709.798366041"
Sep 07 04:00:23.807185 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:23.807138    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:23.907323 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:23.907290    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:24.035082 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:24.035045    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:24.316046 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:24.316015    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:24.583854 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:24.583737    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:27.261830 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:27.261799    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:27.356536 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:27.356503    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:27.586011 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:27.585904    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:27.882485 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:27.882398    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:28.369390 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:28.369355    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"]
Sep 07 04:00:28.372565 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:28.372541    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"
Sep 07 04:00:28.375149 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:28.375123    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-64f80f33-serving-cert\""
Sep 07 04:00:28.384618 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:28.384592    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"]
Sep 07 04:00:28.409051 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:28.408999    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:28.470841 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:28.470805    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec5018ad-e441-43e1-9b3f-62b686bd00b2-openshift-service-ca\") pod \"maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4\" (UID: \"ec5018ad-e441-43e1-9b3f-62b686bd00b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"
Sep 07 04:00:28.470841 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:28.470841    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpt98\" (UniqueName: \"kubernetes.io/projected/ec5018ad-e441-43e1-9b3f-62b686bd00b2-kube-api-access-dpt98\") pod \"maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4\" (UID: \"ec5018ad-e441-43e1-9b3f-62b686bd00b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"
Sep 07 04:00:28.471061 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:28.470914    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec5018ad-e441-43e1-9b3f-62b686bd00b2-maas-api-tls\") pod \"maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4\" (UID: \"ec5018ad-e441-43e1-9b3f-62b686bd00b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"
Sep 07 04:00:28.471061 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:28.471034    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec5018ad-e441-43e1-9b3f-62b686bd00b2-metrics-certs\") pod \"maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4\" (UID: \"ec5018ad-e441-43e1-9b3f-62b686bd00b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"
Sep 07 04:00:28.510081 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:28.510053    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:28.572380 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:28.572339    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec5018ad-e441-43e1-9b3f-62b686bd00b2-metrics-certs\") pod \"maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4\" (UID: \"ec5018ad-e441-43e1-9b3f-62b686bd00b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"
Sep 07 04:00:28.572380 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:28.572379    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec5018ad-e441-43e1-9b3f-62b686bd00b2-openshift-service-ca\") pod \"maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4\" (UID: \"ec5018ad-e441-43e1-9b3f-62b686bd00b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"
Sep 07 04:00:28.572591 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:28.572397    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpt98\" (UniqueName: \"kubernetes.io/projected/ec5018ad-e441-43e1-9b3f-62b686bd00b2-kube-api-access-dpt98\") pod \"maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4\" (UID: \"ec5018ad-e441-43e1-9b3f-62b686bd00b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"
Sep 07 04:00:28.572591 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:28.572435    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec5018ad-e441-43e1-9b3f-62b686bd00b2-maas-api-tls\") pod \"maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4\" (UID: \"ec5018ad-e441-43e1-9b3f-62b686bd00b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"
Sep 07 04:00:28.572591 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:00:28.572547    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-serving-cert: secret "maas-api-e2e-ait-64f80f33-serving-cert" not found
Sep 07 04:00:28.572687 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:00:28.572615    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec5018ad-e441-43e1-9b3f-62b686bd00b2-maas-api-tls podName:ec5018ad-e441-43e1-9b3f-62b686bd00b2 nodeName:}" failed. No retries permitted until 2026-09-07 04:00:29.072596593 +0000 UTC m=+716.006077292 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ec5018ad-e441-43e1-9b3f-62b686bd00b2-maas-api-tls") pod "maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4" (UID: "ec5018ad-e441-43e1-9b3f-62b686bd00b2") : secret "maas-api-e2e-ait-64f80f33-serving-cert" not found
Sep 07 04:00:28.573049 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:28.573029    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec5018ad-e441-43e1-9b3f-62b686bd00b2-openshift-service-ca\") pod \"maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4\" (UID: \"ec5018ad-e441-43e1-9b3f-62b686bd00b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"
Sep 07 04:00:28.574964 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:28.574943    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec5018ad-e441-43e1-9b3f-62b686bd00b2-metrics-certs\") pod \"maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4\" (UID: \"ec5018ad-e441-43e1-9b3f-62b686bd00b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"
Sep 07 04:00:28.584921 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:28.584887    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpt98\" (UniqueName: \"kubernetes.io/projected/ec5018ad-e441-43e1-9b3f-62b686bd00b2-kube-api-access-dpt98\") pod \"maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4\" (UID: \"ec5018ad-e441-43e1-9b3f-62b686bd00b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"
Sep 07 04:00:28.766024 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:28.765973    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:28.858533 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:28.858499    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"
Sep 07 04:00:28.875481 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:28.875437    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:28.969531 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:28.969504    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:29.025244 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:29.025158    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh"]
Sep 07 04:00:29.028198 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:29.028169    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh"
Sep 07 04:00:29.030628 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:29.030605    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-64f80f33-dockercfg-rpvfk\""
Sep 07 04:00:29.030759 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:29.030639    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-64f80f33\""
Sep 07 04:00:29.037733 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:29.037708    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh"]
Sep 07 04:00:29.077076 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:29.077037    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec5018ad-e441-43e1-9b3f-62b686bd00b2-maas-api-tls\") pod \"maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4\" (UID: \"ec5018ad-e441-43e1-9b3f-62b686bd00b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"
Sep 07 04:00:29.079742 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:29.079718    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec5018ad-e441-43e1-9b3f-62b686bd00b2-maas-api-tls\") pod \"maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4\" (UID: \"ec5018ad-e441-43e1-9b3f-62b686bd00b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"
Sep 07 04:00:29.178060 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:29.178016    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8907a72-9612-4ec5-9099-d1094225ac15-plugins-config-volume\") pod \"payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh\" (UID: \"b8907a72-9612-4ec5-9099-d1094225ac15\") " pod="openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh"
Sep 07 04:00:29.178282 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:29.178159    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcl5v\" (UniqueName: \"kubernetes.io/projected/b8907a72-9612-4ec5-9099-d1094225ac15-kube-api-access-jcl5v\") pod \"payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh\" (UID: \"b8907a72-9612-4ec5-9099-d1094225ac15\") " pod="openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh"
Sep 07 04:00:29.279523 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:29.279432    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8907a72-9612-4ec5-9099-d1094225ac15-plugins-config-volume\") pod \"payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh\" (UID: \"b8907a72-9612-4ec5-9099-d1094225ac15\") " pod="openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh"
Sep 07 04:00:29.279678 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:29.279537    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcl5v\" (UniqueName: \"kubernetes.io/projected/b8907a72-9612-4ec5-9099-d1094225ac15-kube-api-access-jcl5v\") pod \"payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh\" (UID: \"b8907a72-9612-4ec5-9099-d1094225ac15\") " pod="openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh"
Sep 07 04:00:29.280150 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:29.280128    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8907a72-9612-4ec5-9099-d1094225ac15-plugins-config-volume\") pod \"payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh\" (UID: \"b8907a72-9612-4ec5-9099-d1094225ac15\") " pod="openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh"
Sep 07 04:00:29.284357 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:29.284335    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"
Sep 07 04:00:29.287604 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:29.287579    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcl5v\" (UniqueName: \"kubernetes.io/projected/b8907a72-9612-4ec5-9099-d1094225ac15-kube-api-access-jcl5v\") pod \"payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh\" (UID: \"b8907a72-9612-4ec5-9099-d1094225ac15\") " pod="openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh"
Sep 07 04:00:29.340497 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:29.340464    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh"
Sep 07 04:00:29.558799 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:29.558771    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"]
Sep 07 04:00:29.561115 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:00:29.561091    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec5018ad_e441_43e1_9b3f_62b686bd00b2.slice/crio-c7df1d3053b70210f9aab299901d41ddf908e9c73467e864ce812d84cb74423b WatchSource:0}: Error finding container c7df1d3053b70210f9aab299901d41ddf908e9c73467e864ce812d84cb74423b: Status 404 returned error can't find the container with id c7df1d3053b70210f9aab299901d41ddf908e9c73467e864ce812d84cb74423b
Sep 07 04:00:29.618274 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:29.618252    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh"]
Sep 07 04:00:29.620820 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:00:29.620791    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8907a72_9612_4ec5_9099_d1094225ac15.slice/crio-69646158a8a3b7ea49b225cf988dc213962c844f07e27854bc1fb0317060b647 WatchSource:0}: Error finding container 69646158a8a3b7ea49b225cf988dc213962c844f07e27854bc1fb0317060b647: Status 404 returned error can't find the container with id 69646158a8a3b7ea49b225cf988dc213962c844f07e27854bc1fb0317060b647
Sep 07 04:00:29.886136 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:29.886046    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4" event={"ID":"ec5018ad-e441-43e1-9b3f-62b686bd00b2","Type":"ContainerStarted","Data":"c7df1d3053b70210f9aab299901d41ddf908e9c73467e864ce812d84cb74423b"}
Sep 07 04:00:29.887620 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:29.887597    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh" event={"ID":"b8907a72-9612-4ec5-9099-d1094225ac15","Type":"ContainerStarted","Data":"8b7df15535fb936a9f833b1c092c2f956717bf99db8ba907f49f5bccabe6aaf5"}
Sep 07 04:00:29.887762 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:29.887634    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh" event={"ID":"b8907a72-9612-4ec5-9099-d1094225ac15","Type":"ContainerStarted","Data":"69646158a8a3b7ea49b225cf988dc213962c844f07e27854bc1fb0317060b647"}
Sep 07 04:00:29.887762 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:29.887728    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh"
Sep 07 04:00:29.904233 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:29.904163    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh" podStartSLOduration=0.904145039 podStartE2EDuration="904.145039ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:00:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:00:29.9029726 +0000 UTC m=+716.836453329" watchObservedRunningTime="2026-09-07 04:00:29.904145039 +0000 UTC m=+716.837625756"
Sep 07 04:00:30.555255 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:30.555204    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:30.654236 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:30.654176    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:30.894775 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:30.894676    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4" event={"ID":"ec5018ad-e441-43e1-9b3f-62b686bd00b2","Type":"ContainerStarted","Data":"b03d993d045ece4f8088f78bd57501c02f0b9b289169641226d7ccd0b0535445"}
Sep 07 04:00:30.894775 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:30.894753    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"
Sep 07 04:00:30.922952 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:30.922880    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4" podStartSLOduration=2.547741027 podStartE2EDuration="2.922856488s" totalImagesPullingTime="375.115461ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:00:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:00:30.914430182 +0000 UTC m=+717.847910899" watchObservedRunningTime="2026-09-07 04:00:30.922856488 +0000 UTC m=+717.856337189"
Sep 07 04:00:31.691155 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:31.691118    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"]
Sep 07 04:00:31.694856 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:31.694828    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"
Sep 07 04:00:31.697345 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:31.697321    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-908959-serving-cert\""
Sep 07 04:00:31.707527 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:31.707503    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"]
Sep 07 04:00:31.731114 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:31.731079    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:31.805557 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:31.805517    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spjlx\" (UniqueName: \"kubernetes.io/projected/b40461de-9411-481b-991b-2c68ef901f6d-kube-api-access-spjlx\") pod \"maas-api-e2e-autores-908959-84b6c69c9-q2h9n\" (UID: \"b40461de-9411-481b-991b-2c68ef901f6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"
Sep 07 04:00:31.805740 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:31.805596    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b40461de-9411-481b-991b-2c68ef901f6d-maas-api-tls\") pod \"maas-api-e2e-autores-908959-84b6c69c9-q2h9n\" (UID: \"b40461de-9411-481b-991b-2c68ef901f6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"
Sep 07 04:00:31.805740 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:31.805627    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b40461de-9411-481b-991b-2c68ef901f6d-openshift-service-ca\") pod \"maas-api-e2e-autores-908959-84b6c69c9-q2h9n\" (UID: \"b40461de-9411-481b-991b-2c68ef901f6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"
Sep 07 04:00:31.805740 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:31.805642    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b40461de-9411-481b-991b-2c68ef901f6d-metrics-certs\") pod \"maas-api-e2e-autores-908959-84b6c69c9-q2h9n\" (UID: \"b40461de-9411-481b-991b-2c68ef901f6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"
Sep 07 04:00:31.825971 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:31.825929    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:31.906298 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:31.906267    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spjlx\" (UniqueName: \"kubernetes.io/projected/b40461de-9411-481b-991b-2c68ef901f6d-kube-api-access-spjlx\") pod \"maas-api-e2e-autores-908959-84b6c69c9-q2h9n\" (UID: \"b40461de-9411-481b-991b-2c68ef901f6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"
Sep 07 04:00:31.906481 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:31.906343    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b40461de-9411-481b-991b-2c68ef901f6d-maas-api-tls\") pod \"maas-api-e2e-autores-908959-84b6c69c9-q2h9n\" (UID: \"b40461de-9411-481b-991b-2c68ef901f6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"
Sep 07 04:00:31.906481 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:31.906375    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b40461de-9411-481b-991b-2c68ef901f6d-openshift-service-ca\") pod \"maas-api-e2e-autores-908959-84b6c69c9-q2h9n\" (UID: \"b40461de-9411-481b-991b-2c68ef901f6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"
Sep 07 04:00:31.906481 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:31.906392    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b40461de-9411-481b-991b-2c68ef901f6d-metrics-certs\") pod \"maas-api-e2e-autores-908959-84b6c69c9-q2h9n\" (UID: \"b40461de-9411-481b-991b-2c68ef901f6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"
Sep 07 04:00:31.907182 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:31.907161    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b40461de-9411-481b-991b-2c68ef901f6d-openshift-service-ca\") pod \"maas-api-e2e-autores-908959-84b6c69c9-q2h9n\" (UID: \"b40461de-9411-481b-991b-2c68ef901f6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"
Sep 07 04:00:31.908932 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:31.908912    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b40461de-9411-481b-991b-2c68ef901f6d-maas-api-tls\") pod \"maas-api-e2e-autores-908959-84b6c69c9-q2h9n\" (UID: \"b40461de-9411-481b-991b-2c68ef901f6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"
Sep 07 04:00:31.909031 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:31.908963    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b40461de-9411-481b-991b-2c68ef901f6d-metrics-certs\") pod \"maas-api-e2e-autores-908959-84b6c69c9-q2h9n\" (UID: \"b40461de-9411-481b-991b-2c68ef901f6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"
Sep 07 04:00:31.914978 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:31.914952    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spjlx\" (UniqueName: \"kubernetes.io/projected/b40461de-9411-481b-991b-2c68ef901f6d-kube-api-access-spjlx\") pod \"maas-api-e2e-autores-908959-84b6c69c9-q2h9n\" (UID: \"b40461de-9411-481b-991b-2c68ef901f6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"
Sep 07 04:00:31.949275 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:31.949170    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:32.006407 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:32.006369    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"
Sep 07 04:00:32.289697 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:32.289670    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"]
Sep 07 04:00:32.291947 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:00:32.291923    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb40461de_9411_481b_991b_2c68ef901f6d.slice/crio-f7b846592e7aadd989f2ef582a099d9d9096d5cd1c2e88f2cbc4f4a9e6f63b58 WatchSource:0}: Error finding container f7b846592e7aadd989f2ef582a099d9d9096d5cd1c2e88f2cbc4f4a9e6f63b58: Status 404 returned error can't find the container with id f7b846592e7aadd989f2ef582a099d9d9096d5cd1c2e88f2cbc4f4a9e6f63b58
Sep 07 04:00:32.312873 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:32.312841    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:32.402222 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:32.402187    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:32.907226 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:32.907124    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n" event={"ID":"b40461de-9411-481b-991b-2c68ef901f6d","Type":"ContainerStarted","Data":"70456bfb6bb83d52fb1114afadbdef9da959de6b052679659ed39bc750bb1d3d"}
Sep 07 04:00:32.907226 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:32.907173    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n" event={"ID":"b40461de-9411-481b-991b-2c68ef901f6d","Type":"ContainerStarted","Data":"f7b846592e7aadd989f2ef582a099d9d9096d5cd1c2e88f2cbc4f4a9e6f63b58"}
Sep 07 04:00:32.907657 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:32.907247    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"
Sep 07 04:00:32.924988 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:32.924938    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n" podStartSLOduration=1.609728805 podStartE2EDuration="1.924918323s" totalImagesPullingTime="315.189518ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:00:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:00:32.924565181 +0000 UTC m=+719.858045900" watchObservedRunningTime="2026-09-07 04:00:32.924918323 +0000 UTC m=+719.858399040"
Sep 07 04:00:34.970039 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:34.970004    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:35.116432 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:35.116234    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:35.212065 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:35.212028    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:35.311635 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:35.311525    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:36.904884 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:36.904856    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"
Sep 07 04:00:37.052177 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:37.052142    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:37.151405 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:37.151374    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:37.252312 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:37.252282    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:37.353398 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:37.353363    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:38.039318 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:38.039283    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:38.917490 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:38.917410    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"
Sep 07 04:00:39.503980 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:39.503943    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:40.897382 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:40.897347    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh"
Sep 07 04:00:42.105511 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:42.105470    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:43.774100 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:43.774062    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:43.879836 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:43.879798    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:43.974558 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:43.974525    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:44.113543 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:44.113463    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:44.651490 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:44.651446    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:44.986746 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:44.986713    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:45.353042 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:45.352961    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:45.616170 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:45.616079    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:48.073450 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:48.073413    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:48.172858 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:48.172823    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:49.191096 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.191064    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"]
Sep 07 04:00:49.196081 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.196058    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"
Sep 07 04:00:49.198654 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.198626    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-a8da4e-serving-cert\""
Sep 07 04:00:49.207241 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.207195    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"]
Sep 07 04:00:49.226251 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.226224    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:49.261443 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.261408    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hprw2\" (UniqueName: \"kubernetes.io/projected/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-kube-api-access-hprw2\") pod \"maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx\" (UID: \"4b3fbc08-1c66-42e3-b3c6-bdb34726c317\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"
Sep 07 04:00:49.261620 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.261449    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-metrics-certs\") pod \"maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx\" (UID: \"4b3fbc08-1c66-42e3-b3c6-bdb34726c317\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"
Sep 07 04:00:49.261620 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.261473    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx\" (UID: \"4b3fbc08-1c66-42e3-b3c6-bdb34726c317\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"
Sep 07 04:00:49.261620 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.261546    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-maas-api-tls\") pod \"maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx\" (UID: \"4b3fbc08-1c66-42e3-b3c6-bdb34726c317\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"
Sep 07 04:00:49.285522 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.285488    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:49.362464 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.362424    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-metrics-certs\") pod \"maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx\" (UID: \"4b3fbc08-1c66-42e3-b3c6-bdb34726c317\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"
Sep 07 04:00:49.362464 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.362462    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx\" (UID: \"4b3fbc08-1c66-42e3-b3c6-bdb34726c317\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"
Sep 07 04:00:49.362676 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.362493    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-maas-api-tls\") pod \"maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx\" (UID: \"4b3fbc08-1c66-42e3-b3c6-bdb34726c317\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"
Sep 07 04:00:49.362676 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.362616    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hprw2\" (UniqueName: \"kubernetes.io/projected/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-kube-api-access-hprw2\") pod \"maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx\" (UID: \"4b3fbc08-1c66-42e3-b3c6-bdb34726c317\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"
Sep 07 04:00:49.362749 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:00:49.362675    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-serving-cert: secret "maas-api-e2e-inf-b-a8da4e-serving-cert" not found
Sep 07 04:00:49.362784 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:00:49.362754    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-maas-api-tls podName:4b3fbc08-1c66-42e3-b3c6-bdb34726c317 nodeName:}" failed. No retries permitted until 2026-09-07 04:00:49.862735873 +0000 UTC m=+736.796216573 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-maas-api-tls") pod "maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx" (UID: "4b3fbc08-1c66-42e3-b3c6-bdb34726c317") : secret "maas-api-e2e-inf-b-a8da4e-serving-cert" not found
Sep 07 04:00:49.363264 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.363248    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx\" (UID: \"4b3fbc08-1c66-42e3-b3c6-bdb34726c317\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"
Sep 07 04:00:49.365030 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.365011    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-metrics-certs\") pod \"maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx\" (UID: \"4b3fbc08-1c66-42e3-b3c6-bdb34726c317\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"
Sep 07 04:00:49.373473 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.373448    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hprw2\" (UniqueName: \"kubernetes.io/projected/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-kube-api-access-hprw2\") pod \"maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx\" (UID: \"4b3fbc08-1c66-42e3-b3c6-bdb34726c317\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"
Sep 07 04:00:49.725071 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.725011    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:49.825038 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.824998    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:49.867324 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.867296    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-maas-api-tls\") pod \"maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx\" (UID: \"4b3fbc08-1c66-42e3-b3c6-bdb34726c317\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"
Sep 07 04:00:49.867486 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.867391    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b"]
Sep 07 04:00:49.867486 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:00:49.867470    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-serving-cert: secret "maas-api-e2e-inf-b-a8da4e-serving-cert" not found
Sep 07 04:00:49.867561 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:00:49.867541    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-maas-api-tls podName:4b3fbc08-1c66-42e3-b3c6-bdb34726c317 nodeName:}" failed. No retries permitted until 2026-09-07 04:00:50.867522286 +0000 UTC m=+737.801002985 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-maas-api-tls") pod "maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx" (UID: "4b3fbc08-1c66-42e3-b3c6-bdb34726c317") : secret "maas-api-e2e-inf-b-a8da4e-serving-cert" not found
Sep 07 04:00:49.871060 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.871042    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b"
Sep 07 04:00:49.873980 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.873956    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-a8da4e\""
Sep 07 04:00:49.874102 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.874014    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-a8da4e-dockercfg-wwlbj\""
Sep 07 04:00:49.884907 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.884877    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b"]
Sep 07 04:00:49.968597 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.968564    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f664f7ca-4a00-45f0-8017-0858b5479076-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b\" (UID: \"f664f7ca-4a00-45f0-8017-0858b5479076\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b"
Sep 07 04:00:49.968782 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:49.968652    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88fb5\" (UniqueName: \"kubernetes.io/projected/f664f7ca-4a00-45f0-8017-0858b5479076-kube-api-access-88fb5\") pod \"payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b\" (UID: \"f664f7ca-4a00-45f0-8017-0858b5479076\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b"
Sep 07 04:00:50.011435 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:50.011350    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:50.069428 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:50.069396    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f664f7ca-4a00-45f0-8017-0858b5479076-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b\" (UID: \"f664f7ca-4a00-45f0-8017-0858b5479076\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b"
Sep 07 04:00:50.069658 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:50.069452    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88fb5\" (UniqueName: \"kubernetes.io/projected/f664f7ca-4a00-45f0-8017-0858b5479076-kube-api-access-88fb5\") pod \"payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b\" (UID: \"f664f7ca-4a00-45f0-8017-0858b5479076\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b"
Sep 07 04:00:50.070443 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:50.070423    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f664f7ca-4a00-45f0-8017-0858b5479076-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b\" (UID: \"f664f7ca-4a00-45f0-8017-0858b5479076\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b"
Sep 07 04:00:50.079090 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:50.079067    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88fb5\" (UniqueName: \"kubernetes.io/projected/f664f7ca-4a00-45f0-8017-0858b5479076-kube-api-access-88fb5\") pod \"payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b\" (UID: \"f664f7ca-4a00-45f0-8017-0858b5479076\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b"
Sep 07 04:00:50.181809 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:50.181773    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b"
Sep 07 04:00:50.302758 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:50.302670    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:50.465887 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:50.465844    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b"]
Sep 07 04:00:50.468311 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:00:50.468266    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf664f7ca_4a00_45f0_8017_0858b5479076.slice/crio-1061cff5b2a0b0d0a7890ea78bfac20aec0d32c03867f83d24b7752be765cd53 WatchSource:0}: Error finding container 1061cff5b2a0b0d0a7890ea78bfac20aec0d32c03867f83d24b7752be765cd53: Status 404 returned error can't find the container with id 1061cff5b2a0b0d0a7890ea78bfac20aec0d32c03867f83d24b7752be765cd53
Sep 07 04:00:50.879627 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:50.879593    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-maas-api-tls\") pod \"maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx\" (UID: \"4b3fbc08-1c66-42e3-b3c6-bdb34726c317\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"
Sep 07 04:00:50.882173 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:50.882146    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-maas-api-tls\") pod \"maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx\" (UID: \"4b3fbc08-1c66-42e3-b3c6-bdb34726c317\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"
Sep 07 04:00:50.998723 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:50.998684    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b" event={"ID":"f664f7ca-4a00-45f0-8017-0858b5479076","Type":"ContainerStarted","Data":"c943d232b8b96fb933bc7530eed85f8d458d60fc40e4b3f703c54ed112e88fe0"}
Sep 07 04:00:50.998907 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:50.998872    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b"
Sep 07 04:00:50.998907 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:50.998894    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b" event={"ID":"f664f7ca-4a00-45f0-8017-0858b5479076","Type":"ContainerStarted","Data":"1061cff5b2a0b0d0a7890ea78bfac20aec0d32c03867f83d24b7752be765cd53"}
Sep 07 04:00:51.008950 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:51.008915    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"
Sep 07 04:00:51.016386 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:51.016328    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b" podStartSLOduration=2.016311773 podStartE2EDuration="2.016311773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:00:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:00:51.013585974 +0000 UTC m=+737.947066693" watchObservedRunningTime="2026-09-07 04:00:51.016311773 +0000 UTC m=+737.949792491"
Sep 07 04:00:51.018357 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:51.018329    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:51.288091 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:51.288064    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"]
Sep 07 04:00:51.289963 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:00:51.289929    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b3fbc08_1c66_42e3_b3c6_bdb34726c317.slice/crio-97ed250cff6f17025532ef805f1fea350b21fbea6e6ef126a47fad2991730a77 WatchSource:0}: Error finding container 97ed250cff6f17025532ef805f1fea350b21fbea6e6ef126a47fad2991730a77: Status 404 returned error can't find the container with id 97ed250cff6f17025532ef805f1fea350b21fbea6e6ef126a47fad2991730a77
Sep 07 04:00:51.342625 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:51.342596    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:51.441938 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:51.441903    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:51.733937 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:51.733904    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:52.005738 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:52.005696    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx" event={"ID":"4b3fbc08-1c66-42e3-b3c6-bdb34726c317","Type":"ContainerStarted","Data":"7e3a71326024e1ae8c084b284805cfb8cba4a65e9b3176620aa3009310c213c0"}
Sep 07 04:00:52.005966 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:52.005766    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx" event={"ID":"4b3fbc08-1c66-42e3-b3c6-bdb34726c317","Type":"ContainerStarted","Data":"97ed250cff6f17025532ef805f1fea350b21fbea6e6ef126a47fad2991730a77"}
Sep 07 04:00:52.006040 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:52.005968    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"
Sep 07 04:00:52.024100 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:52.024050    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx" podStartSLOduration=2.665098032 podStartE2EDuration="3.024034257s" totalImagesPullingTime="358.936225ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:00:52.022114278 +0000 UTC m=+738.955595011" watchObservedRunningTime="2026-09-07 04:00:52.024034257 +0000 UTC m=+738.957514974"
Sep 07 04:00:52.871540 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:52.871498    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:53.495516 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:53.495484    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:54.096869 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:54.096835    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:55.361826 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:55.361791    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:00:58.016700 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:00:58.016667    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"
Sep 07 04:01:02.008334 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:02.008303    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b"
Sep 07 04:01:07.001667 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:07.001635    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:08.641867 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:08.641830    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:09.429153 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.429123    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"]
Sep 07 04:01:09.429460 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.429417    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4" podUID="ec5018ad-e441-43e1-9b3f-62b686bd00b2" containerName="maas-api" containerID="cri-o://b03d993d045ece4f8088f78bd57501c02f0b9b289169641226d7ccd0b0535445" gracePeriod=30
Sep 07 04:01:09.447126 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.447084    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:09.477563 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.477533    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh"]
Sep 07 04:01:09.477890 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.477855    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh" podUID="b8907a72-9612-4ec5-9099-d1094225ac15" containerName="payload-processing" containerID="cri-o://8b7df15535fb936a9f833b1c092c2f956717bf99db8ba907f49f5bccabe6aaf5" gracePeriod=30
Sep 07 04:01:09.546847 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.546816    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:09.622308 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.622166    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:09.699006 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.698961    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:09.753852 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.753826    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"
Sep 07 04:01:09.756783 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.756757    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh"
Sep 07 04:01:09.871503 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.871470    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec5018ad-e441-43e1-9b3f-62b686bd00b2-maas-api-tls\") pod \"ec5018ad-e441-43e1-9b3f-62b686bd00b2\" (UID: \"ec5018ad-e441-43e1-9b3f-62b686bd00b2\") "
Sep 07 04:01:09.871503 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.871504    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec5018ad-e441-43e1-9b3f-62b686bd00b2-metrics-certs\") pod \"ec5018ad-e441-43e1-9b3f-62b686bd00b2\" (UID: \"ec5018ad-e441-43e1-9b3f-62b686bd00b2\") "
Sep 07 04:01:09.871718 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.871554    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec5018ad-e441-43e1-9b3f-62b686bd00b2-openshift-service-ca\") pod \"ec5018ad-e441-43e1-9b3f-62b686bd00b2\" (UID: \"ec5018ad-e441-43e1-9b3f-62b686bd00b2\") "
Sep 07 04:01:09.871718 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.871575    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpt98\" (UniqueName: \"kubernetes.io/projected/ec5018ad-e441-43e1-9b3f-62b686bd00b2-kube-api-access-dpt98\") pod \"ec5018ad-e441-43e1-9b3f-62b686bd00b2\" (UID: \"ec5018ad-e441-43e1-9b3f-62b686bd00b2\") "
Sep 07 04:01:09.871718 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.871594    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcl5v\" (UniqueName: \"kubernetes.io/projected/b8907a72-9612-4ec5-9099-d1094225ac15-kube-api-access-jcl5v\") pod \"b8907a72-9612-4ec5-9099-d1094225ac15\" (UID: \"b8907a72-9612-4ec5-9099-d1094225ac15\") "
Sep 07 04:01:09.871718 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.871678    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8907a72-9612-4ec5-9099-d1094225ac15-plugins-config-volume\") pod \"b8907a72-9612-4ec5-9099-d1094225ac15\" (UID: \"b8907a72-9612-4ec5-9099-d1094225ac15\") "
Sep 07 04:01:09.871949 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.871921    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec5018ad-e441-43e1-9b3f-62b686bd00b2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ec5018ad-e441-43e1-9b3f-62b686bd00b2" (UID: "ec5018ad-e441-43e1-9b3f-62b686bd00b2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 04:01:09.872160 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.872132    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8907a72-9612-4ec5-9099-d1094225ac15-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b8907a72-9612-4ec5-9099-d1094225ac15" (UID: "b8907a72-9612-4ec5-9099-d1094225ac15"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 04:01:09.874040 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.874005    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec5018ad-e441-43e1-9b3f-62b686bd00b2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ec5018ad-e441-43e1-9b3f-62b686bd00b2" (UID: "ec5018ad-e441-43e1-9b3f-62b686bd00b2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:01:09.874140 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.874048    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec5018ad-e441-43e1-9b3f-62b686bd00b2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ec5018ad-e441-43e1-9b3f-62b686bd00b2" (UID: "ec5018ad-e441-43e1-9b3f-62b686bd00b2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:01:09.874140 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.874074    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec5018ad-e441-43e1-9b3f-62b686bd00b2-kube-api-access-dpt98" (OuterVolumeSpecName: "kube-api-access-dpt98") pod "ec5018ad-e441-43e1-9b3f-62b686bd00b2" (UID: "ec5018ad-e441-43e1-9b3f-62b686bd00b2"). InnerVolumeSpecName "kube-api-access-dpt98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:01:09.874140 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.874093    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8907a72-9612-4ec5-9099-d1094225ac15-kube-api-access-jcl5v" (OuterVolumeSpecName: "kube-api-access-jcl5v") pod "b8907a72-9612-4ec5-9099-d1094225ac15" (UID: "b8907a72-9612-4ec5-9099-d1094225ac15"). InnerVolumeSpecName "kube-api-access-jcl5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:01:09.875844 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.875816    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:09.973024 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.972929    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpt98\" (UniqueName: \"kubernetes.io/projected/ec5018ad-e441-43e1-9b3f-62b686bd00b2-kube-api-access-dpt98\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:01:09.973024 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.972962    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcl5v\" (UniqueName: \"kubernetes.io/projected/b8907a72-9612-4ec5-9099-d1094225ac15-kube-api-access-jcl5v\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:01:09.973024 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.972974    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8907a72-9612-4ec5-9099-d1094225ac15-plugins-config-volume\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:01:09.973024 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.972987    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec5018ad-e441-43e1-9b3f-62b686bd00b2-maas-api-tls\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:01:09.973024 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.972998    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec5018ad-e441-43e1-9b3f-62b686bd00b2-metrics-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:01:09.973024 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:09.973008    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec5018ad-e441-43e1-9b3f-62b686bd00b2-openshift-service-ca\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:01:10.104155 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:10.104122    2583 generic.go:356] "Generic (PLEG): container finished" podID="ec5018ad-e441-43e1-9b3f-62b686bd00b2" containerID="b03d993d045ece4f8088f78bd57501c02f0b9b289169641226d7ccd0b0535445" exitCode=0
Sep 07 04:01:10.104390 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:10.104166    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"
Sep 07 04:01:10.104390 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:10.104249    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4" event={"ID":"ec5018ad-e441-43e1-9b3f-62b686bd00b2","Type":"ContainerDied","Data":"b03d993d045ece4f8088f78bd57501c02f0b9b289169641226d7ccd0b0535445"}
Sep 07 04:01:10.104390 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:10.104306    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4" event={"ID":"ec5018ad-e441-43e1-9b3f-62b686bd00b2","Type":"ContainerDied","Data":"c7df1d3053b70210f9aab299901d41ddf908e9c73467e864ce812d84cb74423b"}
Sep 07 04:01:10.104390 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:10.104339    2583 scope.go:117] "RemoveContainer" containerID="b03d993d045ece4f8088f78bd57501c02f0b9b289169641226d7ccd0b0535445"
Sep 07 04:01:10.105680 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:10.105659    2583 generic.go:356] "Generic (PLEG): container finished" podID="b8907a72-9612-4ec5-9099-d1094225ac15" containerID="8b7df15535fb936a9f833b1c092c2f956717bf99db8ba907f49f5bccabe6aaf5" exitCode=0
Sep 07 04:01:10.105839 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:10.105696    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh"
Sep 07 04:01:10.105839 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:10.105695    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh" event={"ID":"b8907a72-9612-4ec5-9099-d1094225ac15","Type":"ContainerDied","Data":"8b7df15535fb936a9f833b1c092c2f956717bf99db8ba907f49f5bccabe6aaf5"}
Sep 07 04:01:10.105839 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:10.105816    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh" event={"ID":"b8907a72-9612-4ec5-9099-d1094225ac15","Type":"ContainerDied","Data":"69646158a8a3b7ea49b225cf988dc213962c844f07e27854bc1fb0317060b647"}
Sep 07 04:01:10.115635 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:10.115616    2583 scope.go:117] "RemoveContainer" containerID="b03d993d045ece4f8088f78bd57501c02f0b9b289169641226d7ccd0b0535445"
Sep 07 04:01:10.115954 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:01:10.115930    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b03d993d045ece4f8088f78bd57501c02f0b9b289169641226d7ccd0b0535445\": container with ID starting with b03d993d045ece4f8088f78bd57501c02f0b9b289169641226d7ccd0b0535445 not found: ID does not exist" containerID="b03d993d045ece4f8088f78bd57501c02f0b9b289169641226d7ccd0b0535445"
Sep 07 04:01:10.116129 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:10.115962    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b03d993d045ece4f8088f78bd57501c02f0b9b289169641226d7ccd0b0535445"} err="failed to get container status \"b03d993d045ece4f8088f78bd57501c02f0b9b289169641226d7ccd0b0535445\": rpc error: code = NotFound desc = could not find container \"b03d993d045ece4f8088f78bd57501c02f0b9b289169641226d7ccd0b0535445\": container with ID starting with b03d993d045ece4f8088f78bd57501c02f0b9b289169641226d7ccd0b0535445 not found: ID does not exist"
Sep 07 04:01:10.116129 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:10.115979    2583 scope.go:117] "RemoveContainer" containerID="8b7df15535fb936a9f833b1c092c2f956717bf99db8ba907f49f5bccabe6aaf5"
Sep 07 04:01:10.125389 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:10.125369    2583 scope.go:117] "RemoveContainer" containerID="8b7df15535fb936a9f833b1c092c2f956717bf99db8ba907f49f5bccabe6aaf5"
Sep 07 04:01:10.125675 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:01:10.125657    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b7df15535fb936a9f833b1c092c2f956717bf99db8ba907f49f5bccabe6aaf5\": container with ID starting with 8b7df15535fb936a9f833b1c092c2f956717bf99db8ba907f49f5bccabe6aaf5 not found: ID does not exist" containerID="8b7df15535fb936a9f833b1c092c2f956717bf99db8ba907f49f5bccabe6aaf5"
Sep 07 04:01:10.125741 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:10.125681    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b7df15535fb936a9f833b1c092c2f956717bf99db8ba907f49f5bccabe6aaf5"} err="failed to get container status \"8b7df15535fb936a9f833b1c092c2f956717bf99db8ba907f49f5bccabe6aaf5\": rpc error: code = NotFound desc = could not find container \"8b7df15535fb936a9f833b1c092c2f956717bf99db8ba907f49f5bccabe6aaf5\": container with ID starting with 8b7df15535fb936a9f833b1c092c2f956717bf99db8ba907f49f5bccabe6aaf5 not found: ID does not exist"
Sep 07 04:01:10.132802 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:10.132776    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"]
Sep 07 04:01:10.135916 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:10.135893    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-64f80f33-6b6d4ccb94-9xcf4"]
Sep 07 04:01:10.146910 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:10.146880    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh"]
Sep 07 04:01:10.150636 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:10.150610    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-64f80f33-85996c85f5-5mnxh"]
Sep 07 04:01:10.226205 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:10.226120    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:10.323917 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:10.323875    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:10.431300 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:10.431267    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:10.524001 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:10.523967    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:10.713633 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:10.713602    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:11.044281 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:11.044242    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:11.523582 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:11.523544    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8907a72-9612-4ec5-9099-d1094225ac15" path="/var/lib/kubelet/pods/b8907a72-9612-4ec5-9099-d1094225ac15/volumes"
Sep 07 04:01:11.524142 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:11.524124    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec5018ad-e441-43e1-9b3f-62b686bd00b2" path="/var/lib/kubelet/pods/ec5018ad-e441-43e1-9b3f-62b686bd00b2/volumes"
Sep 07 04:01:12.905646 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:12.905608    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:13.272090 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.272053    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:13.378402 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.378373    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"]
Sep 07 04:01:13.378901 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.378844    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8907a72-9612-4ec5-9099-d1094225ac15" containerName="payload-processing"
Sep 07 04:01:13.378901 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.378904    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8907a72-9612-4ec5-9099-d1094225ac15" containerName="payload-processing"
Sep 07 04:01:13.379049 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.378939    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec5018ad-e441-43e1-9b3f-62b686bd00b2" containerName="maas-api"
Sep 07 04:01:13.379049 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.378948    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec5018ad-e441-43e1-9b3f-62b686bd00b2" containerName="maas-api"
Sep 07 04:01:13.379049 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.379030    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec5018ad-e441-43e1-9b3f-62b686bd00b2" containerName="maas-api"
Sep 07 04:01:13.379049 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.379041    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8907a72-9612-4ec5-9099-d1094225ac15" containerName="payload-processing"
Sep 07 04:01:13.380441 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.380418    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:13.383177 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.383160    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"
Sep 07 04:01:13.385873 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.385854    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-e1915828-serving-cert\""
Sep 07 04:01:13.392461 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.391732    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"]
Sep 07 04:01:13.475367 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.475338    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:13.505619 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.505582    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e027ea61-d5e4-4191-9613-21ca4df8ed34-maas-api-tls\") pod \"maas-api-e2e-migrate-e1915828-556f58f5db-tm858\" (UID: \"e027ea61-d5e4-4191-9613-21ca4df8ed34\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"
Sep 07 04:01:13.505836 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.505673    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e027ea61-d5e4-4191-9613-21ca4df8ed34-openshift-service-ca\") pod \"maas-api-e2e-migrate-e1915828-556f58f5db-tm858\" (UID: \"e027ea61-d5e4-4191-9613-21ca4df8ed34\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"
Sep 07 04:01:13.505836 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.505732    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e027ea61-d5e4-4191-9613-21ca4df8ed34-metrics-certs\") pod \"maas-api-e2e-migrate-e1915828-556f58f5db-tm858\" (UID: \"e027ea61-d5e4-4191-9613-21ca4df8ed34\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"
Sep 07 04:01:13.505836 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.505759    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlmxt\" (UniqueName: \"kubernetes.io/projected/e027ea61-d5e4-4191-9613-21ca4df8ed34-kube-api-access-rlmxt\") pod \"maas-api-e2e-migrate-e1915828-556f58f5db-tm858\" (UID: \"e027ea61-d5e4-4191-9613-21ca4df8ed34\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"
Sep 07 04:01:13.607317 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.607244    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e027ea61-d5e4-4191-9613-21ca4df8ed34-openshift-service-ca\") pod \"maas-api-e2e-migrate-e1915828-556f58f5db-tm858\" (UID: \"e027ea61-d5e4-4191-9613-21ca4df8ed34\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"
Sep 07 04:01:13.607478 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.607326    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e027ea61-d5e4-4191-9613-21ca4df8ed34-metrics-certs\") pod \"maas-api-e2e-migrate-e1915828-556f58f5db-tm858\" (UID: \"e027ea61-d5e4-4191-9613-21ca4df8ed34\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"
Sep 07 04:01:13.607478 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.607359    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlmxt\" (UniqueName: \"kubernetes.io/projected/e027ea61-d5e4-4191-9613-21ca4df8ed34-kube-api-access-rlmxt\") pod \"maas-api-e2e-migrate-e1915828-556f58f5db-tm858\" (UID: \"e027ea61-d5e4-4191-9613-21ca4df8ed34\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"
Sep 07 04:01:13.607478 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.607444    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e027ea61-d5e4-4191-9613-21ca4df8ed34-maas-api-tls\") pod \"maas-api-e2e-migrate-e1915828-556f58f5db-tm858\" (UID: \"e027ea61-d5e4-4191-9613-21ca4df8ed34\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"
Sep 07 04:01:13.607659 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:01:13.607603    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-serving-cert: secret "maas-api-e2e-migrate-e1915828-serving-cert" not found
Sep 07 04:01:13.607713 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:01:13.607663    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e027ea61-d5e4-4191-9613-21ca4df8ed34-maas-api-tls podName:e027ea61-d5e4-4191-9613-21ca4df8ed34 nodeName:}" failed. No retries permitted until 2026-09-07 04:01:14.107645716 +0000 UTC m=+761.041126412 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e027ea61-d5e4-4191-9613-21ca4df8ed34-maas-api-tls") pod "maas-api-e2e-migrate-e1915828-556f58f5db-tm858" (UID: "e027ea61-d5e4-4191-9613-21ca4df8ed34") : secret "maas-api-e2e-migrate-e1915828-serving-cert" not found
Sep 07 04:01:13.608044 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.608018    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e027ea61-d5e4-4191-9613-21ca4df8ed34-openshift-service-ca\") pod \"maas-api-e2e-migrate-e1915828-556f58f5db-tm858\" (UID: \"e027ea61-d5e4-4191-9613-21ca4df8ed34\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"
Sep 07 04:01:13.609917 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.609886    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e027ea61-d5e4-4191-9613-21ca4df8ed34-metrics-certs\") pod \"maas-api-e2e-migrate-e1915828-556f58f5db-tm858\" (UID: \"e027ea61-d5e4-4191-9613-21ca4df8ed34\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"
Sep 07 04:01:13.615834 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.615808    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlmxt\" (UniqueName: \"kubernetes.io/projected/e027ea61-d5e4-4191-9613-21ca4df8ed34-kube-api-access-rlmxt\") pod \"maas-api-e2e-migrate-e1915828-556f58f5db-tm858\" (UID: \"e027ea61-d5e4-4191-9613-21ca4df8ed34\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"
Sep 07 04:01:13.794098 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.794061    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:13.893752 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.893665    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:13.990045 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:13.990006    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:14.111696 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:14.111666    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e027ea61-d5e4-4191-9613-21ca4df8ed34-maas-api-tls\") pod \"maas-api-e2e-migrate-e1915828-556f58f5db-tm858\" (UID: \"e027ea61-d5e4-4191-9613-21ca4df8ed34\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"
Sep 07 04:01:14.114114 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:14.114095    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e027ea61-d5e4-4191-9613-21ca4df8ed34-maas-api-tls\") pod \"maas-api-e2e-migrate-e1915828-556f58f5db-tm858\" (UID: \"e027ea61-d5e4-4191-9613-21ca4df8ed34\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"
Sep 07 04:01:14.298259 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:14.298195    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"
Sep 07 04:01:14.577567 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:14.577546    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"]
Sep 07 04:01:14.579978 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:01:14.579948    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode027ea61_d5e4_4191_9613_21ca4df8ed34.slice/crio-1f8ba8fc4e7f8d48762a5b8dfe6010c15cd1ea954be3ff7de944967070cbbb37 WatchSource:0}: Error finding container 1f8ba8fc4e7f8d48762a5b8dfe6010c15cd1ea954be3ff7de944967070cbbb37: Status 404 returned error can't find the container with id 1f8ba8fc4e7f8d48762a5b8dfe6010c15cd1ea954be3ff7de944967070cbbb37
Sep 07 04:01:15.136002 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:15.135971    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858" event={"ID":"e027ea61-d5e4-4191-9613-21ca4df8ed34","Type":"ContainerStarted","Data":"7ef76f7f9586f03c12b91afde79b6c6331b1c7f4ff3076f8b61b83e5af742052"}
Sep 07 04:01:15.136402 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:15.136022    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858" event={"ID":"e027ea61-d5e4-4191-9613-21ca4df8ed34","Type":"ContainerStarted","Data":"1f8ba8fc4e7f8d48762a5b8dfe6010c15cd1ea954be3ff7de944967070cbbb37"}
Sep 07 04:01:15.136402 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:15.136086    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"
Sep 07 04:01:15.155268 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:15.155184    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858" podStartSLOduration=1.753028499 podStartE2EDuration="2.155168856s" totalImagesPullingTime="402.140357ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:01:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:01:15.152567748 +0000 UTC m=+762.086048468" watchObservedRunningTime="2026-09-07 04:01:15.155168856 +0000 UTC m=+762.088649574"
Sep 07 04:01:21.148179 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:21.148101    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"
Sep 07 04:01:25.761657 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.761611    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"]
Sep 07 04:01:25.765440 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.765423    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:25.769335 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.769076    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-908959\"/\"openshift-service-ca.crt\""
Sep 07 04:01:25.769335 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.769132    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-908959\"/\"autores-model-908959-kserve-self-signed-certs\""
Sep 07 04:01:25.769335 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.769184    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-908959\"/\"kube-root-ca.crt\""
Sep 07 04:01:25.769585 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.769411    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-908959\"/\"default-dockercfg-czt2z\""
Sep 07 04:01:25.773542 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.773516    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"]
Sep 07 04:01:25.833196 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.833162    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdcsn\" (UniqueName: \"kubernetes.io/projected/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-kube-api-access-pdcsn\") pod \"autores-model-908959-kserve-56db7bbb7f-gmdlg\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") " pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:25.833196 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.833200    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-tls-certs\") pod \"autores-model-908959-kserve-56db7bbb7f-gmdlg\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") " pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:25.833468 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.833316    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-model-cache\") pod \"autores-model-908959-kserve-56db7bbb7f-gmdlg\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") " pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:25.833468 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.833364    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-dshm\") pod \"autores-model-908959-kserve-56db7bbb7f-gmdlg\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") " pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:25.833588 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.833483    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-home\") pod \"autores-model-908959-kserve-56db7bbb7f-gmdlg\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") " pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:25.833588 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.833520    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-tmp-dir\") pod \"autores-model-908959-kserve-56db7bbb7f-gmdlg\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") " pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:25.878399 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.878358    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:25.934117 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.934075    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-model-cache\") pod \"autores-model-908959-kserve-56db7bbb7f-gmdlg\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") " pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:25.934376 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.934141    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-dshm\") pod \"autores-model-908959-kserve-56db7bbb7f-gmdlg\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") " pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:25.934376 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.934192    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-home\") pod \"autores-model-908959-kserve-56db7bbb7f-gmdlg\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") " pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:25.934376 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.934236    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-tmp-dir\") pod \"autores-model-908959-kserve-56db7bbb7f-gmdlg\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") " pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:25.934376 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.934289    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdcsn\" (UniqueName: \"kubernetes.io/projected/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-kube-api-access-pdcsn\") pod \"autores-model-908959-kserve-56db7bbb7f-gmdlg\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") " pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:25.934376 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.934318    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-tls-certs\") pod \"autores-model-908959-kserve-56db7bbb7f-gmdlg\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") " pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:25.934618 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.934532    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-model-cache\") pod \"autores-model-908959-kserve-56db7bbb7f-gmdlg\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") " pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:25.934618 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.934596    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-home\") pod \"autores-model-908959-kserve-56db7bbb7f-gmdlg\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") " pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:25.934693 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.934625    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-tmp-dir\") pod \"autores-model-908959-kserve-56db7bbb7f-gmdlg\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") " pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:25.936825 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.936800    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-dshm\") pod \"autores-model-908959-kserve-56db7bbb7f-gmdlg\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") " pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:25.936984 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.936963    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-tls-certs\") pod \"autores-model-908959-kserve-56db7bbb7f-gmdlg\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") " pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:25.942286 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:25.942260    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdcsn\" (UniqueName: \"kubernetes.io/projected/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-kube-api-access-pdcsn\") pod \"autores-model-908959-kserve-56db7bbb7f-gmdlg\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") " pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:26.077814 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:26.077719    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:26.352366 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:26.352336    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"]
Sep 07 04:01:26.354823 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:01:26.354792    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9ecb0a1_b85f_4afb_b0b0_5a38f49aef0c.slice/crio-37bcd5189217fa6028347c7ab1feca357697ad5c8b5d93dcfcca533cfe84ca7b WatchSource:0}: Error finding container 37bcd5189217fa6028347c7ab1feca357697ad5c8b5d93dcfcca533cfe84ca7b: Status 404 returned error can't find the container with id 37bcd5189217fa6028347c7ab1feca357697ad5c8b5d93dcfcca533cfe84ca7b
Sep 07 04:01:27.198963 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:27.198923    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg" event={"ID":"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c","Type":"ContainerStarted","Data":"58f0eda97694a52d699d1068460a242c091e3177c2be5e3d297419f640fd77a9"}
Sep 07 04:01:27.198963 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:27.198969    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg" event={"ID":"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c","Type":"ContainerStarted","Data":"37bcd5189217fa6028347c7ab1feca357697ad5c8b5d93dcfcca533cfe84ca7b"}
Sep 07 04:01:27.217759 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:27.217695    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg" podStartSLOduration=2.217678093 podStartE2EDuration="2.217678093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:01:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:01:27.215633334 +0000 UTC m=+774.149114067" watchObservedRunningTime="2026-09-07 04:01:27.217678093 +0000 UTC m=+774.151158819"
Sep 07 04:01:29.478758 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:29.478721    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:36.078787 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:36.078751    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:36.079304 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:36.078809    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:36.091627 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:36.091599    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:36.262135 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:36.262105    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:01:44.811614 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:44.811582    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"]
Sep 07 04:01:44.812277 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:44.811786    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858" podUID="e027ea61-d5e4-4191-9613-21ca4df8ed34" containerName="maas-api" containerID="cri-o://7ef76f7f9586f03c12b91afde79b6c6331b1c7f4ff3076f8b61b83e5af742052" gracePeriod=30
Sep 07 04:01:44.841582 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:44.841544    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:44.903890 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:44.903858    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:45.004632 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.004588    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:45.063370 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.063287    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"
Sep 07 04:01:45.063370 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.063322    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:45.152407 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.152373    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:45.230331 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.230298    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e027ea61-d5e4-4191-9613-21ca4df8ed34-metrics-certs\") pod \"e027ea61-d5e4-4191-9613-21ca4df8ed34\" (UID: \"e027ea61-d5e4-4191-9613-21ca4df8ed34\") "
Sep 07 04:01:45.230515 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.230402    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlmxt\" (UniqueName: \"kubernetes.io/projected/e027ea61-d5e4-4191-9613-21ca4df8ed34-kube-api-access-rlmxt\") pod \"e027ea61-d5e4-4191-9613-21ca4df8ed34\" (UID: \"e027ea61-d5e4-4191-9613-21ca4df8ed34\") "
Sep 07 04:01:45.230515 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.230436    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e027ea61-d5e4-4191-9613-21ca4df8ed34-openshift-service-ca\") pod \"e027ea61-d5e4-4191-9613-21ca4df8ed34\" (UID: \"e027ea61-d5e4-4191-9613-21ca4df8ed34\") "
Sep 07 04:01:45.230515 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.230468    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e027ea61-d5e4-4191-9613-21ca4df8ed34-maas-api-tls\") pod \"e027ea61-d5e4-4191-9613-21ca4df8ed34\" (UID: \"e027ea61-d5e4-4191-9613-21ca4df8ed34\") "
Sep 07 04:01:45.230810 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.230784    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e027ea61-d5e4-4191-9613-21ca4df8ed34-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e027ea61-d5e4-4191-9613-21ca4df8ed34" (UID: "e027ea61-d5e4-4191-9613-21ca4df8ed34"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 04:01:45.232627 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.232606    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e027ea61-d5e4-4191-9613-21ca4df8ed34-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e027ea61-d5e4-4191-9613-21ca4df8ed34" (UID: "e027ea61-d5e4-4191-9613-21ca4df8ed34"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:01:45.232741 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.232642    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e027ea61-d5e4-4191-9613-21ca4df8ed34-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e027ea61-d5e4-4191-9613-21ca4df8ed34" (UID: "e027ea61-d5e4-4191-9613-21ca4df8ed34"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:01:45.232741 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.232661    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e027ea61-d5e4-4191-9613-21ca4df8ed34-kube-api-access-rlmxt" (OuterVolumeSpecName: "kube-api-access-rlmxt") pod "e027ea61-d5e4-4191-9613-21ca4df8ed34" (UID: "e027ea61-d5e4-4191-9613-21ca4df8ed34"). InnerVolumeSpecName "kube-api-access-rlmxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:01:45.297190 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.297156    2583 generic.go:356] "Generic (PLEG): container finished" podID="e027ea61-d5e4-4191-9613-21ca4df8ed34" containerID="7ef76f7f9586f03c12b91afde79b6c6331b1c7f4ff3076f8b61b83e5af742052" exitCode=0
Sep 07 04:01:45.297386 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.297201    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"
Sep 07 04:01:45.297386 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.297245    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858" event={"ID":"e027ea61-d5e4-4191-9613-21ca4df8ed34","Type":"ContainerDied","Data":"7ef76f7f9586f03c12b91afde79b6c6331b1c7f4ff3076f8b61b83e5af742052"}
Sep 07 04:01:45.297386 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.297300    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858" event={"ID":"e027ea61-d5e4-4191-9613-21ca4df8ed34","Type":"ContainerDied","Data":"1f8ba8fc4e7f8d48762a5b8dfe6010c15cd1ea954be3ff7de944967070cbbb37"}
Sep 07 04:01:45.297386 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.297330    2583 scope.go:117] "RemoveContainer" containerID="7ef76f7f9586f03c12b91afde79b6c6331b1c7f4ff3076f8b61b83e5af742052"
Sep 07 04:01:45.307676 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.307657    2583 scope.go:117] "RemoveContainer" containerID="7ef76f7f9586f03c12b91afde79b6c6331b1c7f4ff3076f8b61b83e5af742052"
Sep 07 04:01:45.308004 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:01:45.307987    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ef76f7f9586f03c12b91afde79b6c6331b1c7f4ff3076f8b61b83e5af742052\": container with ID starting with 7ef76f7f9586f03c12b91afde79b6c6331b1c7f4ff3076f8b61b83e5af742052 not found: ID does not exist" containerID="7ef76f7f9586f03c12b91afde79b6c6331b1c7f4ff3076f8b61b83e5af742052"
Sep 07 04:01:45.308066 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.308016    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ef76f7f9586f03c12b91afde79b6c6331b1c7f4ff3076f8b61b83e5af742052"} err="failed to get container status \"7ef76f7f9586f03c12b91afde79b6c6331b1c7f4ff3076f8b61b83e5af742052\": rpc error: code = NotFound desc = could not find container \"7ef76f7f9586f03c12b91afde79b6c6331b1c7f4ff3076f8b61b83e5af742052\": container with ID starting with 7ef76f7f9586f03c12b91afde79b6c6331b1c7f4ff3076f8b61b83e5af742052 not found: ID does not exist"
Sep 07 04:01:45.318744 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.318668    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"]
Sep 07 04:01:45.322863 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.322837    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e1915828-556f58f5db-tm858"]
Sep 07 04:01:45.331372 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.331347    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e027ea61-d5e4-4191-9613-21ca4df8ed34-metrics-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:01:45.331372 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.331371    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlmxt\" (UniqueName: \"kubernetes.io/projected/e027ea61-d5e4-4191-9613-21ca4df8ed34-kube-api-access-rlmxt\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:01:45.331515 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.331380    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e027ea61-d5e4-4191-9613-21ca4df8ed34-openshift-service-ca\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:01:45.331515 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.331392    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e027ea61-d5e4-4191-9613-21ca4df8ed34-maas-api-tls\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:01:45.521537 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.521502    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e027ea61-d5e4-4191-9613-21ca4df8ed34" path="/var/lib/kubelet/pods/e027ea61-d5e4-4191-9613-21ca4df8ed34/volumes"
Sep 07 04:01:45.716982 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.716952    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:45.932984 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:45.932930    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:46.107003 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:46.106908    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:47.858807 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:47.858770    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"]
Sep 07 04:01:47.859485 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:47.859455    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e027ea61-d5e4-4191-9613-21ca4df8ed34" containerName="maas-api"
Sep 07 04:01:47.859485 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:47.859475    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e027ea61-d5e4-4191-9613-21ca4df8ed34" containerName="maas-api"
Sep 07 04:01:47.859630 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:47.859541    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e027ea61-d5e4-4191-9613-21ca4df8ed34" containerName="maas-api"
Sep 07 04:01:47.864260 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:47.864246    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:01:47.866597 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:47.866579    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-1038cd\"/\"default-dockercfg-5h9lp\""
Sep 07 04:01:47.867695 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:47.867677    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-1038cd\"/\"kube-root-ca.crt\""
Sep 07 04:01:47.867788 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:47.867681    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-1038cd\"/\"openshift-service-ca.crt\""
Sep 07 04:01:47.867788 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:47.867738    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-1038cd\"/\"test-model-1038cd-kserve-self-signed-certs\""
Sep 07 04:01:47.871981 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:47.871831    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"]
Sep 07 04:01:47.955408 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:47.955367    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-model-cache\") pod \"test-model-1038cd-kserve-7bb94f64cf-wmqp8\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") " pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:01:47.955408 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:47.955410    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-dshm\") pod \"test-model-1038cd-kserve-7bb94f64cf-wmqp8\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") " pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:01:47.955618 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:47.955442    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b20abe23-755e-49f6-a042-2146dd85933e-tls-certs\") pod \"test-model-1038cd-kserve-7bb94f64cf-wmqp8\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") " pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:01:47.955618 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:47.955550    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-tmp-dir\") pod \"test-model-1038cd-kserve-7bb94f64cf-wmqp8\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") " pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:01:47.955618 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:47.955582    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfcv6\" (UniqueName: \"kubernetes.io/projected/b20abe23-755e-49f6-a042-2146dd85933e-kube-api-access-tfcv6\") pod \"test-model-1038cd-kserve-7bb94f64cf-wmqp8\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") " pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:01:47.955761 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:47.955669    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-home\") pod \"test-model-1038cd-kserve-7bb94f64cf-wmqp8\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") " pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:01:48.056378 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:48.056335    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-model-cache\") pod \"test-model-1038cd-kserve-7bb94f64cf-wmqp8\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") " pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:01:48.056378 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:48.056380    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-dshm\") pod \"test-model-1038cd-kserve-7bb94f64cf-wmqp8\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") " pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:01:48.056617 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:48.056409    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b20abe23-755e-49f6-a042-2146dd85933e-tls-certs\") pod \"test-model-1038cd-kserve-7bb94f64cf-wmqp8\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") " pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:01:48.056617 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:48.056467    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-tmp-dir\") pod \"test-model-1038cd-kserve-7bb94f64cf-wmqp8\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") " pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:01:48.056617 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:48.056490    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfcv6\" (UniqueName: \"kubernetes.io/projected/b20abe23-755e-49f6-a042-2146dd85933e-kube-api-access-tfcv6\") pod \"test-model-1038cd-kserve-7bb94f64cf-wmqp8\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") " pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:01:48.056617 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:48.056536    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-home\") pod \"test-model-1038cd-kserve-7bb94f64cf-wmqp8\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") " pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:01:48.056783 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:48.056739    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-model-cache\") pod \"test-model-1038cd-kserve-7bb94f64cf-wmqp8\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") " pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:01:48.057086 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:48.057061    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-tmp-dir\") pod \"test-model-1038cd-kserve-7bb94f64cf-wmqp8\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") " pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:01:48.057252 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:48.057171    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-home\") pod \"test-model-1038cd-kserve-7bb94f64cf-wmqp8\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") " pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:01:48.059041 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:48.059015    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-dshm\") pod \"test-model-1038cd-kserve-7bb94f64cf-wmqp8\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") " pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:01:48.059154 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:48.059111    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b20abe23-755e-49f6-a042-2146dd85933e-tls-certs\") pod \"test-model-1038cd-kserve-7bb94f64cf-wmqp8\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") " pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:01:48.065304 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:48.065285    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfcv6\" (UniqueName: \"kubernetes.io/projected/b20abe23-755e-49f6-a042-2146dd85933e-kube-api-access-tfcv6\") pod \"test-model-1038cd-kserve-7bb94f64cf-wmqp8\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") " pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:01:48.077424 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:48.077398    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:48.174357 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:48.174263    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:01:48.450536 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:48.450503    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"]
Sep 07 04:01:48.452645 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:01:48.452616    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb20abe23_755e_49f6_a042_2146dd85933e.slice/crio-4868e7a98bd68c687b935df641dc4b9e691343be7cb7d7bc9540d44df2dfdf36 WatchSource:0}: Error finding container 4868e7a98bd68c687b935df641dc4b9e691343be7cb7d7bc9540d44df2dfdf36: Status 404 returned error can't find the container with id 4868e7a98bd68c687b935df641dc4b9e691343be7cb7d7bc9540d44df2dfdf36
Sep 07 04:01:48.904513 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:48.904433    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:49.003606 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:49.003574    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:49.322557 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:49.322516    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8" event={"ID":"b20abe23-755e-49f6-a042-2146dd85933e","Type":"ContainerStarted","Data":"9b4a3d231bb410e0c33d44700c28ce6693d3fbe86e0dff17c6362214ee30ec58"}
Sep 07 04:01:49.322557 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:49.322573    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8" event={"ID":"b20abe23-755e-49f6-a042-2146dd85933e","Type":"ContainerStarted","Data":"4868e7a98bd68c687b935df641dc4b9e691343be7cb7d7bc9540d44df2dfdf36"}
Sep 07 04:01:49.341167 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:49.341098    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8" podStartSLOduration=2.341075293 podStartE2EDuration="2.341075293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:01:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:01:49.340306252 +0000 UTC m=+796.273786972" watchObservedRunningTime="2026-09-07 04:01:49.341075293 +0000 UTC m=+796.274556012"
Sep 07 04:01:50.037935 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:50.037898    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:50.098993 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:50.098933    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:50.199572 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:50.199540    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:50.536933 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:50.536901    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:50.602530 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:50.602496    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:52.454287 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:52.454253    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"]
Sep 07 04:01:52.475277 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:52.475248    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:52.677128 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:52.677089    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:56.877330 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:56.877241    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:01:58.175361 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:58.175308    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:01:58.175878 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:58.175465    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:01:58.188332 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:58.188303    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:01:58.387248 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:01:58.387199    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:02:03.051953 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.051921    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"]
Sep 07 04:02:03.052462 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.052168    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n" podUID="b40461de-9411-481b-991b-2c68ef901f6d" containerName="maas-api" containerID="cri-o://70456bfb6bb83d52fb1114afadbdef9da959de6b052679659ed39bc750bb1d3d" gracePeriod=30
Sep 07 04:02:03.077963 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.077927    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:03.191515 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.191487    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:03.263473 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.263429    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:03.317620 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.317554    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"
Sep 07 04:02:03.396966 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.396932    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spjlx\" (UniqueName: \"kubernetes.io/projected/b40461de-9411-481b-991b-2c68ef901f6d-kube-api-access-spjlx\") pod \"b40461de-9411-481b-991b-2c68ef901f6d\" (UID: \"b40461de-9411-481b-991b-2c68ef901f6d\") "
Sep 07 04:02:03.397155 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.397009    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b40461de-9411-481b-991b-2c68ef901f6d-openshift-service-ca\") pod \"b40461de-9411-481b-991b-2c68ef901f6d\" (UID: \"b40461de-9411-481b-991b-2c68ef901f6d\") "
Sep 07 04:02:03.397155 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.397055    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b40461de-9411-481b-991b-2c68ef901f6d-metrics-certs\") pod \"b40461de-9411-481b-991b-2c68ef901f6d\" (UID: \"b40461de-9411-481b-991b-2c68ef901f6d\") "
Sep 07 04:02:03.397155 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.397111    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b40461de-9411-481b-991b-2c68ef901f6d-maas-api-tls\") pod \"b40461de-9411-481b-991b-2c68ef901f6d\" (UID: \"b40461de-9411-481b-991b-2c68ef901f6d\") "
Sep 07 04:02:03.397430 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.397396    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b40461de-9411-481b-991b-2c68ef901f6d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b40461de-9411-481b-991b-2c68ef901f6d" (UID: "b40461de-9411-481b-991b-2c68ef901f6d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 04:02:03.399427 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.399397    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b40461de-9411-481b-991b-2c68ef901f6d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b40461de-9411-481b-991b-2c68ef901f6d" (UID: "b40461de-9411-481b-991b-2c68ef901f6d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:02:03.399427 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.399402    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b40461de-9411-481b-991b-2c68ef901f6d-kube-api-access-spjlx" (OuterVolumeSpecName: "kube-api-access-spjlx") pod "b40461de-9411-481b-991b-2c68ef901f6d" (UID: "b40461de-9411-481b-991b-2c68ef901f6d"). InnerVolumeSpecName "kube-api-access-spjlx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:02:03.399562 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.399483    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b40461de-9411-481b-991b-2c68ef901f6d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b40461de-9411-481b-991b-2c68ef901f6d" (UID: "b40461de-9411-481b-991b-2c68ef901f6d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:02:03.402971 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.402946    2583 generic.go:356] "Generic (PLEG): container finished" podID="b40461de-9411-481b-991b-2c68ef901f6d" containerID="70456bfb6bb83d52fb1114afadbdef9da959de6b052679659ed39bc750bb1d3d" exitCode=0
Sep 07 04:02:03.403065 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.402993    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"
Sep 07 04:02:03.403065 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.403029    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n" event={"ID":"b40461de-9411-481b-991b-2c68ef901f6d","Type":"ContainerDied","Data":"70456bfb6bb83d52fb1114afadbdef9da959de6b052679659ed39bc750bb1d3d"}
Sep 07 04:02:03.403168 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.403092    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n" event={"ID":"b40461de-9411-481b-991b-2c68ef901f6d","Type":"ContainerDied","Data":"f7b846592e7aadd989f2ef582a099d9d9096d5cd1c2e88f2cbc4f4a9e6f63b58"}
Sep 07 04:02:03.403168 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.403125    2583 scope.go:117] "RemoveContainer" containerID="70456bfb6bb83d52fb1114afadbdef9da959de6b052679659ed39bc750bb1d3d"
Sep 07 04:02:03.419645 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.419621    2583 scope.go:117] "RemoveContainer" containerID="70456bfb6bb83d52fb1114afadbdef9da959de6b052679659ed39bc750bb1d3d"
Sep 07 04:02:03.420025 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:02:03.420005    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70456bfb6bb83d52fb1114afadbdef9da959de6b052679659ed39bc750bb1d3d\": container with ID starting with 70456bfb6bb83d52fb1114afadbdef9da959de6b052679659ed39bc750bb1d3d not found: ID does not exist" containerID="70456bfb6bb83d52fb1114afadbdef9da959de6b052679659ed39bc750bb1d3d"
Sep 07 04:02:03.420128 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.420037    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70456bfb6bb83d52fb1114afadbdef9da959de6b052679659ed39bc750bb1d3d"} err="failed to get container status \"70456bfb6bb83d52fb1114afadbdef9da959de6b052679659ed39bc750bb1d3d\": rpc error: code = NotFound desc = could not find container \"70456bfb6bb83d52fb1114afadbdef9da959de6b052679659ed39bc750bb1d3d\": container with ID starting with 70456bfb6bb83d52fb1114afadbdef9da959de6b052679659ed39bc750bb1d3d not found: ID does not exist"
Sep 07 04:02:03.430921 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.430884    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"]
Sep 07 04:02:03.434235 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.434190    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-908959-84b6c69c9-q2h9n"]
Sep 07 04:02:03.498612 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.498574    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spjlx\" (UniqueName: \"kubernetes.io/projected/b40461de-9411-481b-991b-2c68ef901f6d-kube-api-access-spjlx\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:02:03.498612 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.498604    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b40461de-9411-481b-991b-2c68ef901f6d-openshift-service-ca\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:02:03.498612 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.498615    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b40461de-9411-481b-991b-2c68ef901f6d-metrics-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:02:03.498612 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.498623    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b40461de-9411-481b-991b-2c68ef901f6d-maas-api-tls\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:02:03.523199 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.523166    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b40461de-9411-481b-991b-2c68ef901f6d" path="/var/lib/kubelet/pods/b40461de-9411-481b-991b-2c68ef901f6d/volumes"
Sep 07 04:02:03.625870 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:03.625769    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:07.478947 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:07.478906    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg" podUID="f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c" containerName="main" containerID="cri-o://58f0eda97694a52d699d1068460a242c091e3177c2be5e3d297419f640fd77a9" gracePeriod=45
Sep 07 04:02:07.736756 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:07.736694    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:02:07.839996 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:07.839963    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-model-cache\") pod \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") "
Sep 07 04:02:07.840177 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:07.840004    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-tmp-dir\") pod \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") "
Sep 07 04:02:07.840177 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:07.840045    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdcsn\" (UniqueName: \"kubernetes.io/projected/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-kube-api-access-pdcsn\") pod \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") "
Sep 07 04:02:07.840177 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:07.840068    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-tls-certs\") pod \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") "
Sep 07 04:02:07.840177 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:07.840089    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-home\") pod \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") "
Sep 07 04:02:07.840177 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:07.840155    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-dshm\") pod \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\" (UID: \"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c\") "
Sep 07 04:02:07.840494 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:07.840274    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-model-cache" (OuterVolumeSpecName: "model-cache") pod "f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c" (UID: "f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:02:07.840494 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:07.840320    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c" (UID: "f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:02:07.840494 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:07.840466    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-home" (OuterVolumeSpecName: "home") pod "f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c" (UID: "f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:02:07.840616 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:07.840565    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-model-cache\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:02:07.840652 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:07.840619    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-tmp-dir\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:02:07.840652 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:07.840632    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-home\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:02:07.842595 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:07.842573    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-dshm" (OuterVolumeSpecName: "dshm") pod "f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c" (UID: "f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:02:07.842673 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:07.842629    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c" (UID: "f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:02:07.842751 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:07.842727    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-kube-api-access-pdcsn" (OuterVolumeSpecName: "kube-api-access-pdcsn") pod "f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c" (UID: "f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c"). InnerVolumeSpecName "kube-api-access-pdcsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:02:07.941858 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:07.941827    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-dshm\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:02:07.941858 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:07.941854    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdcsn\" (UniqueName: \"kubernetes.io/projected/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-kube-api-access-pdcsn\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:02:07.941858 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:07.941864    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c-tls-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:02:08.431858 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:08.431818    2583 generic.go:356] "Generic (PLEG): container finished" podID="f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c" containerID="58f0eda97694a52d699d1068460a242c091e3177c2be5e3d297419f640fd77a9" exitCode=0
Sep 07 04:02:08.431858 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:08.431860    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"
Sep 07 04:02:08.432110 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:08.431863    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg" event={"ID":"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c","Type":"ContainerDied","Data":"58f0eda97694a52d699d1068460a242c091e3177c2be5e3d297419f640fd77a9"}
Sep 07 04:02:08.432110 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:08.431985    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg" event={"ID":"f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c","Type":"ContainerDied","Data":"37bcd5189217fa6028347c7ab1feca357697ad5c8b5d93dcfcca533cfe84ca7b"}
Sep 07 04:02:08.432110 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:08.432006    2583 scope.go:117] "RemoveContainer" containerID="58f0eda97694a52d699d1068460a242c091e3177c2be5e3d297419f640fd77a9"
Sep 07 04:02:08.441756 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:08.441736    2583 scope.go:117] "RemoveContainer" containerID="58f0eda97694a52d699d1068460a242c091e3177c2be5e3d297419f640fd77a9"
Sep 07 04:02:08.442027 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:02:08.442011    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58f0eda97694a52d699d1068460a242c091e3177c2be5e3d297419f640fd77a9\": container with ID starting with 58f0eda97694a52d699d1068460a242c091e3177c2be5e3d297419f640fd77a9 not found: ID does not exist" containerID="58f0eda97694a52d699d1068460a242c091e3177c2be5e3d297419f640fd77a9"
Sep 07 04:02:08.442078 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:08.442036    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58f0eda97694a52d699d1068460a242c091e3177c2be5e3d297419f640fd77a9"} err="failed to get container status \"58f0eda97694a52d699d1068460a242c091e3177c2be5e3d297419f640fd77a9\": rpc error: code = NotFound desc = could not find container \"58f0eda97694a52d699d1068460a242c091e3177c2be5e3d297419f640fd77a9\": container with ID starting with 58f0eda97694a52d699d1068460a242c091e3177c2be5e3d297419f640fd77a9 not found: ID does not exist"
Sep 07 04:02:08.453879 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:08.453854    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"]
Sep 07 04:02:08.455838 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:08.455820    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-908959/autores-model-908959-kserve-56db7bbb7f-gmdlg"]
Sep 07 04:02:09.489904 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:09.489850    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:09.522957 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:09.522926    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c" path="/var/lib/kubelet/pods/f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c/volumes"
Sep 07 04:02:09.546930 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:09.546901    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:09.726029 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:09.725995    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:10.169333 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:10.169291    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:10.457866 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:10.457826    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:10.769151 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:10.769076    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:10.892120 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:10.892085    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:13.175283 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:13.175246    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:13.274628 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:13.274591    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:14.378520 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.378484    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"]
Sep 07 04:02:14.379153 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.379137    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b40461de-9411-481b-991b-2c68ef901f6d" containerName="maas-api"
Sep 07 04:02:14.379200 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.379156    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b40461de-9411-481b-991b-2c68ef901f6d" containerName="maas-api"
Sep 07 04:02:14.379200 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.379174    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c" containerName="main"
Sep 07 04:02:14.379200 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.379183    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c" containerName="main"
Sep 07 04:02:14.379322 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.379291    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9ecb0a1-b85f-4afb-b0b0-5a38f49aef0c" containerName="main"
Sep 07 04:02:14.379322 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.379305    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b40461de-9411-481b-991b-2c68ef901f6d" containerName="maas-api"
Sep 07 04:02:14.383895 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.383871    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"
Sep 07 04:02:14.386622 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.386598    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-a84f90-serving-cert\""
Sep 07 04:02:14.392661 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.392581    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"]
Sep 07 04:02:14.413136 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.413103    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:14.509748 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.509706    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-maas-api-tls\") pod \"maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj\" (UID: \"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"
Sep 07 04:02:14.509949 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.509782    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcnbw\" (UniqueName: \"kubernetes.io/projected/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-kube-api-access-wcnbw\") pod \"maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj\" (UID: \"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"
Sep 07 04:02:14.509949 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.509852    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-metrics-certs\") pod \"maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj\" (UID: \"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"
Sep 07 04:02:14.510066 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.509953    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-openshift-service-ca\") pod \"maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj\" (UID: \"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"
Sep 07 04:02:14.514532 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.514506    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:14.610548 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.610504    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcnbw\" (UniqueName: \"kubernetes.io/projected/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-kube-api-access-wcnbw\") pod \"maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj\" (UID: \"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"
Sep 07 04:02:14.610732 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.610576    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-metrics-certs\") pod \"maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj\" (UID: \"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"
Sep 07 04:02:14.610786 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.610762    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-openshift-service-ca\") pod \"maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj\" (UID: \"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"
Sep 07 04:02:14.610954 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.610929    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-maas-api-tls\") pod \"maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj\" (UID: \"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"
Sep 07 04:02:14.611063 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:02:14.611039    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-serving-cert: secret "maas-api-e2e-explicit-a84f90-serving-cert" not found
Sep 07 04:02:14.611120 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:02:14.611108    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-maas-api-tls podName:1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3 nodeName:}" failed. No retries permitted until 2026-09-07 04:02:15.111087359 +0000 UTC m=+822.044568059 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-maas-api-tls") pod "maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj" (UID: "1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3") : secret "maas-api-e2e-explicit-a84f90-serving-cert" not found
Sep 07 04:02:14.611508 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.611486    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-openshift-service-ca\") pod \"maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj\" (UID: \"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"
Sep 07 04:02:14.613261 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.613242    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-metrics-certs\") pod \"maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj\" (UID: \"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"
Sep 07 04:02:14.619207 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.619183    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcnbw\" (UniqueName: \"kubernetes.io/projected/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-kube-api-access-wcnbw\") pod \"maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj\" (UID: \"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"
Sep 07 04:02:14.816561 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.816505    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:14.954549 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:14.954511    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:15.051235 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:15.051188    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:15.081024 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:15.080933    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j"]
Sep 07 04:02:15.084753 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:15.084729    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j"
Sep 07 04:02:15.087253 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:15.087204    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-a84f90-dockercfg-r74gk\""
Sep 07 04:02:15.087377 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:15.087282    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-a84f90\""
Sep 07 04:02:15.094786 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:15.094761    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j"]
Sep 07 04:02:15.117005 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:15.116973    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-maas-api-tls\") pod \"maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj\" (UID: \"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"
Sep 07 04:02:15.119637 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:15.119612    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-maas-api-tls\") pod \"maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj\" (UID: \"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"
Sep 07 04:02:15.217592 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:15.217545    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9375483-9747-43fa-aba4-6e81fe6e87d3-plugins-config-volume\") pod \"payload-processing-e2e-explicit-a84f90-89b6847d5-4458j\" (UID: \"f9375483-9747-43fa-aba4-6e81fe6e87d3\") " pod="openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j"
Sep 07 04:02:15.217746 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:15.217640    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gthrl\" (UniqueName: \"kubernetes.io/projected/f9375483-9747-43fa-aba4-6e81fe6e87d3-kube-api-access-gthrl\") pod \"payload-processing-e2e-explicit-a84f90-89b6847d5-4458j\" (UID: \"f9375483-9747-43fa-aba4-6e81fe6e87d3\") " pod="openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j"
Sep 07 04:02:15.297196 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:15.297156    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"
Sep 07 04:02:15.313303 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:15.313272    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:15.318419 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:15.318388    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9375483-9747-43fa-aba4-6e81fe6e87d3-plugins-config-volume\") pod \"payload-processing-e2e-explicit-a84f90-89b6847d5-4458j\" (UID: \"f9375483-9747-43fa-aba4-6e81fe6e87d3\") " pod="openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j"
Sep 07 04:02:15.318566 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:15.318456    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gthrl\" (UniqueName: \"kubernetes.io/projected/f9375483-9747-43fa-aba4-6e81fe6e87d3-kube-api-access-gthrl\") pod \"payload-processing-e2e-explicit-a84f90-89b6847d5-4458j\" (UID: \"f9375483-9747-43fa-aba4-6e81fe6e87d3\") " pod="openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j"
Sep 07 04:02:15.319241 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:15.319189    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9375483-9747-43fa-aba4-6e81fe6e87d3-plugins-config-volume\") pod \"payload-processing-e2e-explicit-a84f90-89b6847d5-4458j\" (UID: \"f9375483-9747-43fa-aba4-6e81fe6e87d3\") " pod="openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j"
Sep 07 04:02:15.327471 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:15.327444    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gthrl\" (UniqueName: \"kubernetes.io/projected/f9375483-9747-43fa-aba4-6e81fe6e87d3-kube-api-access-gthrl\") pod \"payload-processing-e2e-explicit-a84f90-89b6847d5-4458j\" (UID: \"f9375483-9747-43fa-aba4-6e81fe6e87d3\") " pod="openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j"
Sep 07 04:02:15.397017 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:15.396979    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j"
Sep 07 04:02:15.570750 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:15.570728    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"]
Sep 07 04:02:15.572571 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:02:15.572549    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fa04728_985e_40e6_b4b5_0c3fb8e4f8a3.slice/crio-932d757d0b25e413bd5e0ffd9e72a669e2f8291cdf012cde2998f8aec8203f5d WatchSource:0}: Error finding container 932d757d0b25e413bd5e0ffd9e72a669e2f8291cdf012cde2998f8aec8203f5d: Status 404 returned error can't find the container with id 932d757d0b25e413bd5e0ffd9e72a669e2f8291cdf012cde2998f8aec8203f5d
Sep 07 04:02:15.670883 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:15.670861    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j"]
Sep 07 04:02:15.673173 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:02:15.673145    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9375483_9747_43fa_aba4_6e81fe6e87d3.slice/crio-dbc03e5944e1a37ced4e653b5a8e3480068fbaa7c564a588dc257a72c6f5f1cf WatchSource:0}: Error finding container dbc03e5944e1a37ced4e653b5a8e3480068fbaa7c564a588dc257a72c6f5f1cf: Status 404 returned error can't find the container with id dbc03e5944e1a37ced4e653b5a8e3480068fbaa7c564a588dc257a72c6f5f1cf
Sep 07 04:02:16.083509 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:16.083470    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:16.476655 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:16.476622    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj" event={"ID":"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3","Type":"ContainerStarted","Data":"fe573be924ae677f299539e331d7853cde2735e6ba4127cd6084bde4430ff6c2"}
Sep 07 04:02:16.477094 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:16.476682    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"
Sep 07 04:02:16.477094 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:16.476705    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj" event={"ID":"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3","Type":"ContainerStarted","Data":"932d757d0b25e413bd5e0ffd9e72a669e2f8291cdf012cde2998f8aec8203f5d"}
Sep 07 04:02:16.478320 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:16.478295    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j" event={"ID":"f9375483-9747-43fa-aba4-6e81fe6e87d3","Type":"ContainerStarted","Data":"90dfd48ff6796e3977407c3cefdbf623f332dee8542b0959ce0098377a64f566"}
Sep 07 04:02:16.478467 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:16.478334    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j" event={"ID":"f9375483-9747-43fa-aba4-6e81fe6e87d3","Type":"ContainerStarted","Data":"dbc03e5944e1a37ced4e653b5a8e3480068fbaa7c564a588dc257a72c6f5f1cf"}
Sep 07 04:02:16.478467 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:16.478417    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j"
Sep 07 04:02:16.493934 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:16.493885    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj" podStartSLOduration=2.102679489 podStartE2EDuration="2.493866152s" totalImagesPullingTime="391.186663ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:02:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:02:16.491974583 +0000 UTC m=+823.425455302" watchObservedRunningTime="2026-09-07 04:02:16.493866152 +0000 UTC m=+823.427346869"
Sep 07 04:02:16.512839 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:16.512778    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j" podStartSLOduration=1.512757999 podStartE2EDuration="1.512757999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:02:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:02:16.510665258 +0000 UTC m=+823.444146006" watchObservedRunningTime="2026-09-07 04:02:16.512757999 +0000 UTC m=+823.446238784"
Sep 07 04:02:16.701928 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:16.701872    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:20.180779 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:20.180746    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:20.688415 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:20.688380    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:20.964533 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:20.964437    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:21.027737 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:21.027699    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:21.965348 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:21.965312    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:22.389866 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:22.389786    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:22.489302 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:22.489276    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"
Sep 07 04:02:22.698358 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:22.698306    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:23.523056 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:23.523018    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:25.700319 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:25.700278    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:25.800457 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:25.800426    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:27.115240 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:27.115153    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:27.215875 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:27.215840    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:27.316769 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:27.316738    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:27.485961 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:27.485933    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j"
Sep 07 04:02:27.553889 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:27.553859    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:27.641355 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:27.641322    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:35.307078 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:35.307037    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:38.111323 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:38.111288    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:46.497386 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:46.497351    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:47.911732 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:47.911699    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:52.290887 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:52.290855    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:53.362918 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:53.362876    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:54.478010 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:54.477976    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:54.608963 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:54.608926    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:56.995986 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:56.995943    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:57.506342 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:57.506313    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:57.571945 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:57.571890    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:57.627660 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:57.627618    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:57.685330 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:57.685297    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:57.835755 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:57.835672    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:58.471850 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:58.471804    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:58.537727 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:58.537699    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:02:58.602481 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:02:58.602450    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:00.021066 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:00.021030    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:04.629640 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:04.629602    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:05.292107 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.292070    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"]
Sep 07 04:03:05.292374 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.292346    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj" podUID="1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3" containerName="maas-api" containerID="cri-o://fe573be924ae677f299539e331d7853cde2735e6ba4127cd6084bde4430ff6c2" gracePeriod=30
Sep 07 04:03:05.314384 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.314353    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:05.334649 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.334618    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j"]
Sep 07 04:03:05.334848 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.334823    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j" podUID="f9375483-9747-43fa-aba4-6e81fe6e87d3" containerName="payload-processing" containerID="cri-o://90dfd48ff6796e3977407c3cefdbf623f332dee8542b0959ce0098377a64f566" gracePeriod=30
Sep 07 04:03:05.415921 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.415879    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:05.487529 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.487498    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:05.560083 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.560005    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:05.585081 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.585056    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"
Sep 07 04:03:05.616847 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.616800    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j"
Sep 07 04:03:05.710709 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.710677    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcnbw\" (UniqueName: \"kubernetes.io/projected/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-kube-api-access-wcnbw\") pod \"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3\" (UID: \"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3\") "
Sep 07 04:03:05.710709 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.710711    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-openshift-service-ca\") pod \"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3\" (UID: \"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3\") "
Sep 07 04:03:05.711168 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.710771    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9375483-9747-43fa-aba4-6e81fe6e87d3-plugins-config-volume\") pod \"f9375483-9747-43fa-aba4-6e81fe6e87d3\" (UID: \"f9375483-9747-43fa-aba4-6e81fe6e87d3\") "
Sep 07 04:03:05.711168 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.710860    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gthrl\" (UniqueName: \"kubernetes.io/projected/f9375483-9747-43fa-aba4-6e81fe6e87d3-kube-api-access-gthrl\") pod \"f9375483-9747-43fa-aba4-6e81fe6e87d3\" (UID: \"f9375483-9747-43fa-aba4-6e81fe6e87d3\") "
Sep 07 04:03:05.711168 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.710911    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-metrics-certs\") pod \"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3\" (UID: \"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3\") "
Sep 07 04:03:05.711168 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.710959    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-maas-api-tls\") pod \"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3\" (UID: \"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3\") "
Sep 07 04:03:05.711417 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.711172    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3" (UID: "1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 04:03:05.711417 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.711258    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9375483-9747-43fa-aba4-6e81fe6e87d3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f9375483-9747-43fa-aba4-6e81fe6e87d3" (UID: "f9375483-9747-43fa-aba4-6e81fe6e87d3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 04:03:05.711417 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.711340    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-openshift-service-ca\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:03:05.711417 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.711354    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9375483-9747-43fa-aba4-6e81fe6e87d3-plugins-config-volume\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:03:05.713068 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.713043    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9375483-9747-43fa-aba4-6e81fe6e87d3-kube-api-access-gthrl" (OuterVolumeSpecName: "kube-api-access-gthrl") pod "f9375483-9747-43fa-aba4-6e81fe6e87d3" (UID: "f9375483-9747-43fa-aba4-6e81fe6e87d3"). InnerVolumeSpecName "kube-api-access-gthrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:03:05.713196 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.713176    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3" (UID: "1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:03:05.713599 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.713572    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-kube-api-access-wcnbw" (OuterVolumeSpecName: "kube-api-access-wcnbw") pod "1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3" (UID: "1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3"). InnerVolumeSpecName "kube-api-access-wcnbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:03:05.713658 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.713600    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3" (UID: "1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:03:05.735422 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.735393    2583 generic.go:356] "Generic (PLEG): container finished" podID="f9375483-9747-43fa-aba4-6e81fe6e87d3" containerID="90dfd48ff6796e3977407c3cefdbf623f332dee8542b0959ce0098377a64f566" exitCode=0
Sep 07 04:03:05.735553 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.735435    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j"
Sep 07 04:03:05.735553 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.735480    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j" event={"ID":"f9375483-9747-43fa-aba4-6e81fe6e87d3","Type":"ContainerDied","Data":"90dfd48ff6796e3977407c3cefdbf623f332dee8542b0959ce0098377a64f566"}
Sep 07 04:03:05.735553 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.735533    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j" event={"ID":"f9375483-9747-43fa-aba4-6e81fe6e87d3","Type":"ContainerDied","Data":"dbc03e5944e1a37ced4e653b5a8e3480068fbaa7c564a588dc257a72c6f5f1cf"}
Sep 07 04:03:05.735553 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.735555    2583 scope.go:117] "RemoveContainer" containerID="90dfd48ff6796e3977407c3cefdbf623f332dee8542b0959ce0098377a64f566"
Sep 07 04:03:05.736914 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.736855    2583 generic.go:356] "Generic (PLEG): container finished" podID="1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3" containerID="fe573be924ae677f299539e331d7853cde2735e6ba4127cd6084bde4430ff6c2" exitCode=0
Sep 07 04:03:05.736914 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.736906    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"
Sep 07 04:03:05.737084 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.736914    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj" event={"ID":"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3","Type":"ContainerDied","Data":"fe573be924ae677f299539e331d7853cde2735e6ba4127cd6084bde4430ff6c2"}
Sep 07 04:03:05.737084 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.736949    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj" event={"ID":"1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3","Type":"ContainerDied","Data":"932d757d0b25e413bd5e0ffd9e72a669e2f8291cdf012cde2998f8aec8203f5d"}
Sep 07 04:03:05.744226 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.744187    2583 scope.go:117] "RemoveContainer" containerID="90dfd48ff6796e3977407c3cefdbf623f332dee8542b0959ce0098377a64f566"
Sep 07 04:03:05.744670 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:03:05.744650    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90dfd48ff6796e3977407c3cefdbf623f332dee8542b0959ce0098377a64f566\": container with ID starting with 90dfd48ff6796e3977407c3cefdbf623f332dee8542b0959ce0098377a64f566 not found: ID does not exist" containerID="90dfd48ff6796e3977407c3cefdbf623f332dee8542b0959ce0098377a64f566"
Sep 07 04:03:05.744761 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.744677    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90dfd48ff6796e3977407c3cefdbf623f332dee8542b0959ce0098377a64f566"} err="failed to get container status \"90dfd48ff6796e3977407c3cefdbf623f332dee8542b0959ce0098377a64f566\": rpc error: code = NotFound desc = could not find container \"90dfd48ff6796e3977407c3cefdbf623f332dee8542b0959ce0098377a64f566\": container with ID starting with 90dfd48ff6796e3977407c3cefdbf623f332dee8542b0959ce0098377a64f566 not found: ID does not exist"
Sep 07 04:03:05.744761 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.744695    2583 scope.go:117] "RemoveContainer" containerID="fe573be924ae677f299539e331d7853cde2735e6ba4127cd6084bde4430ff6c2"
Sep 07 04:03:05.752913 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.752902    2583 scope.go:117] "RemoveContainer" containerID="fe573be924ae677f299539e331d7853cde2735e6ba4127cd6084bde4430ff6c2"
Sep 07 04:03:05.753190 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:03:05.753172    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe573be924ae677f299539e331d7853cde2735e6ba4127cd6084bde4430ff6c2\": container with ID starting with fe573be924ae677f299539e331d7853cde2735e6ba4127cd6084bde4430ff6c2 not found: ID does not exist" containerID="fe573be924ae677f299539e331d7853cde2735e6ba4127cd6084bde4430ff6c2"
Sep 07 04:03:05.753293 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.753198    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe573be924ae677f299539e331d7853cde2735e6ba4127cd6084bde4430ff6c2"} err="failed to get container status \"fe573be924ae677f299539e331d7853cde2735e6ba4127cd6084bde4430ff6c2\": rpc error: code = NotFound desc = could not find container \"fe573be924ae677f299539e331d7853cde2735e6ba4127cd6084bde4430ff6c2\": container with ID starting with fe573be924ae677f299539e331d7853cde2735e6ba4127cd6084bde4430ff6c2 not found: ID does not exist"
Sep 07 04:03:05.760012 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.759990    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"]
Sep 07 04:03:05.763500 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.763472    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a84f90-86d9c96668-7gfzj"]
Sep 07 04:03:05.772976 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.772954    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j"]
Sep 07 04:03:05.777002 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.776976    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a84f90-89b6847d5-4458j"]
Sep 07 04:03:05.812677 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.812597    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gthrl\" (UniqueName: \"kubernetes.io/projected/f9375483-9747-43fa-aba4-6e81fe6e87d3-kube-api-access-gthrl\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:03:05.812677 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.812628    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-metrics-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:03:05.812677 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.812639    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-maas-api-tls\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:03:05.812677 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.812647    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcnbw\" (UniqueName: \"kubernetes.io/projected/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3-kube-api-access-wcnbw\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:03:05.951956 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:05.951920    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:06.070582 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:06.070497    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:06.182883 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:06.182799    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:06.333861 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:06.333783    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:06.434229 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:06.434183    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:06.786716 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:06.786683    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:07.521975 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:07.521945    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3" path="/var/lib/kubelet/pods/1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3/volumes"
Sep 07 04:03:07.522356 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:07.522342    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9375483-9747-43fa-aba4-6e81fe6e87d3" path="/var/lib/kubelet/pods/f9375483-9747-43fa-aba4-6e81fe6e87d3/volumes"
Sep 07 04:03:11.483282 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:11.483250    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:11.583682 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:11.583644    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:11.729245 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:11.729192    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:12.158409 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:12.158373    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:12.489707 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:12.489674    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:12.698192 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:12.698158    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:13.892647 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:13.892575    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:13.989031 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:13.988996    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:14.089650 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:14.089610    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:18.276937 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:18.276897    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:20.443737 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:20.443700    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:32.686579 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:32.686479    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:35.693828 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:35.693790    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:38.274599 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:38.274562    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:38.337038 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:38.337005    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:38.477433 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:38.477402    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:38.595921 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:38.595829    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:44.175773 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:44.175743    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:44.294196 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:44.294169    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:44.500306 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:44.500268    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:44.613713 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:44.613684    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:49.628877 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:49.628827    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:03:54.333375 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:03:54.333340    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:00.067524 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:00.067486    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:00.125395 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:00.125360    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:00.209187 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:00.209149    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:00.315015 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:00.314971    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:03.185868 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:03.185834    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"]
Sep 07 04:04:03.195707 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:03.195658    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:04.201511 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:04.201461    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:04.387269 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:04.387234    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:10.771401 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:10.771368    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:11.503272 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:11.503237    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:14.653179 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:14.653142    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"]
Sep 07 04:04:14.654039 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:14.654006    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt" podUID="f9b44b17-73cc-4c87-8d64-8bcf7bb10f24" containerName="maas-api" containerID="cri-o://a305ea081848210b855c8f0fb2329ddc73e96f1ae61598a5e20d5743d5df0fb5" gracePeriod=30
Sep 07 04:04:14.674203 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:14.674173    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:14.734662 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:14.734624    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:14.797804 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:14.797767    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:14.858418 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:14.858385    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:14.911993 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:14.911930    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"
Sep 07 04:04:14.946634 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:14.946598    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9bhb\" (UniqueName: \"kubernetes.io/projected/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-kube-api-access-p9bhb\") pod \"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24\" (UID: \"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24\") "
Sep 07 04:04:14.946936 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:14.946688    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-metrics-certs\") pod \"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24\" (UID: \"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24\") "
Sep 07 04:04:14.946936 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:14.946773    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-openshift-service-ca\") pod \"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24\" (UID: \"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24\") "
Sep 07 04:04:14.946936 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:14.946803    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-maas-api-tls\") pod \"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24\" (UID: \"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24\") "
Sep 07 04:04:14.947251 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:14.947196    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f9b44b17-73cc-4c87-8d64-8bcf7bb10f24" (UID: "f9b44b17-73cc-4c87-8d64-8bcf7bb10f24"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 04:04:14.949010 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:14.948983    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f9b44b17-73cc-4c87-8d64-8bcf7bb10f24" (UID: "f9b44b17-73cc-4c87-8d64-8bcf7bb10f24"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:04:14.949135 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:14.949003    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f9b44b17-73cc-4c87-8d64-8bcf7bb10f24" (UID: "f9b44b17-73cc-4c87-8d64-8bcf7bb10f24"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:04:14.949135 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:14.949012    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-kube-api-access-p9bhb" (OuterVolumeSpecName: "kube-api-access-p9bhb") pod "f9b44b17-73cc-4c87-8d64-8bcf7bb10f24" (UID: "f9b44b17-73cc-4c87-8d64-8bcf7bb10f24"). InnerVolumeSpecName "kube-api-access-p9bhb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:04:15.048373 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:15.048332    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-metrics-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:04:15.048373 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:15.048366    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-openshift-service-ca\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:04:15.048373 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:15.048379    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-maas-api-tls\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:04:15.048616 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:15.048392    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9bhb\" (UniqueName: \"kubernetes.io/projected/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24-kube-api-access-p9bhb\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:04:15.084900 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:15.084871    2583 generic.go:356] "Generic (PLEG): container finished" podID="f9b44b17-73cc-4c87-8d64-8bcf7bb10f24" containerID="a305ea081848210b855c8f0fb2329ddc73e96f1ae61598a5e20d5743d5df0fb5" exitCode=0
Sep 07 04:04:15.085071 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:15.084917    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"
Sep 07 04:04:15.085071 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:15.084964    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt" event={"ID":"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24","Type":"ContainerDied","Data":"a305ea081848210b855c8f0fb2329ddc73e96f1ae61598a5e20d5743d5df0fb5"}
Sep 07 04:04:15.085071 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:15.085018    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt" event={"ID":"f9b44b17-73cc-4c87-8d64-8bcf7bb10f24","Type":"ContainerDied","Data":"09e36c437ff0d5ff487bb9ebcf349d675440f0a6fc4920840e854f78947de592"}
Sep 07 04:04:15.085071 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:15.085042    2583 scope.go:117] "RemoveContainer" containerID="a305ea081848210b855c8f0fb2329ddc73e96f1ae61598a5e20d5743d5df0fb5"
Sep 07 04:04:15.094943 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:15.094927    2583 scope.go:117] "RemoveContainer" containerID="a305ea081848210b855c8f0fb2329ddc73e96f1ae61598a5e20d5743d5df0fb5"
Sep 07 04:04:15.095264 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:04:15.095244    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a305ea081848210b855c8f0fb2329ddc73e96f1ae61598a5e20d5743d5df0fb5\": container with ID starting with a305ea081848210b855c8f0fb2329ddc73e96f1ae61598a5e20d5743d5df0fb5 not found: ID does not exist" containerID="a305ea081848210b855c8f0fb2329ddc73e96f1ae61598a5e20d5743d5df0fb5"
Sep 07 04:04:15.095362 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:15.095271    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a305ea081848210b855c8f0fb2329ddc73e96f1ae61598a5e20d5743d5df0fb5"} err="failed to get container status \"a305ea081848210b855c8f0fb2329ddc73e96f1ae61598a5e20d5743d5df0fb5\": rpc error: code = NotFound desc = could not find container \"a305ea081848210b855c8f0fb2329ddc73e96f1ae61598a5e20d5743d5df0fb5\": container with ID starting with a305ea081848210b855c8f0fb2329ddc73e96f1ae61598a5e20d5743d5df0fb5 not found: ID does not exist"
Sep 07 04:04:15.107093 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:15.107067    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"]
Sep 07 04:04:15.110928 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:15.110892    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1038cd-5b7dd9969d-8tqwt"]
Sep 07 04:04:15.192328 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:15.192298    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:15.447767 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:15.447673    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:15.521947 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:15.521904    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9b44b17-73cc-4c87-8d64-8bcf7bb10f24" path="/var/lib/kubelet/pods/f9b44b17-73cc-4c87-8d64-8bcf7bb10f24/volumes"
Sep 07 04:04:17.327412 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:17.327373    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:17.431732 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:17.431699    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:17.527793 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:17.527736    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:17.628058 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:17.627955    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:18.206591 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:18.206537    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8" podUID="b20abe23-755e-49f6-a042-2146dd85933e" containerName="main" containerID="cri-o://9b4a3d231bb410e0c33d44700c28ce6693d3fbe86e0dff17c6362214ee30ec58" gracePeriod=45
Sep 07 04:04:18.239618 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:18.239588    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:18.338708 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:18.338679    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:18.439194 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:18.439160    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:18.464267 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:18.464181    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:04:18.582502 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:18.582462    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-dshm\") pod \"b20abe23-755e-49f6-a042-2146dd85933e\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") "
Sep 07 04:04:18.582696 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:18.582522    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-model-cache\") pod \"b20abe23-755e-49f6-a042-2146dd85933e\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") "
Sep 07 04:04:18.582696 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:18.582567    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-home\") pod \"b20abe23-755e-49f6-a042-2146dd85933e\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") "
Sep 07 04:04:18.582696 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:18.582586    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b20abe23-755e-49f6-a042-2146dd85933e-tls-certs\") pod \"b20abe23-755e-49f6-a042-2146dd85933e\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") "
Sep 07 04:04:18.582696 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:18.582634    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfcv6\" (UniqueName: \"kubernetes.io/projected/b20abe23-755e-49f6-a042-2146dd85933e-kube-api-access-tfcv6\") pod \"b20abe23-755e-49f6-a042-2146dd85933e\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") "
Sep 07 04:04:18.582696 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:18.582654    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-tmp-dir\") pod \"b20abe23-755e-49f6-a042-2146dd85933e\" (UID: \"b20abe23-755e-49f6-a042-2146dd85933e\") "
Sep 07 04:04:18.582961 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:18.582813    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-home" (OuterVolumeSpecName: "home") pod "b20abe23-755e-49f6-a042-2146dd85933e" (UID: "b20abe23-755e-49f6-a042-2146dd85933e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:04:18.582961 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:18.582828    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-model-cache" (OuterVolumeSpecName: "model-cache") pod "b20abe23-755e-49f6-a042-2146dd85933e" (UID: "b20abe23-755e-49f6-a042-2146dd85933e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:04:18.583074 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:18.583040    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b20abe23-755e-49f6-a042-2146dd85933e" (UID: "b20abe23-755e-49f6-a042-2146dd85933e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:04:18.583124 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:18.583079    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-home\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:04:18.583124 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:18.583098    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-model-cache\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:04:18.585034 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:18.585003    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b20abe23-755e-49f6-a042-2146dd85933e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b20abe23-755e-49f6-a042-2146dd85933e" (UID: "b20abe23-755e-49f6-a042-2146dd85933e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:04:18.585034 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:18.585006    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-dshm" (OuterVolumeSpecName: "dshm") pod "b20abe23-755e-49f6-a042-2146dd85933e" (UID: "b20abe23-755e-49f6-a042-2146dd85933e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:04:18.585200 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:18.585034    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b20abe23-755e-49f6-a042-2146dd85933e-kube-api-access-tfcv6" (OuterVolumeSpecName: "kube-api-access-tfcv6") pod "b20abe23-755e-49f6-a042-2146dd85933e" (UID: "b20abe23-755e-49f6-a042-2146dd85933e"). InnerVolumeSpecName "kube-api-access-tfcv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:04:18.684613 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:18.684579    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfcv6\" (UniqueName: \"kubernetes.io/projected/b20abe23-755e-49f6-a042-2146dd85933e-kube-api-access-tfcv6\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:04:18.684613 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:18.684611    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-tmp-dir\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:04:18.684816 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:18.684625    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b20abe23-755e-49f6-a042-2146dd85933e-dshm\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:04:18.684816 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:18.684636    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b20abe23-755e-49f6-a042-2146dd85933e-tls-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:04:19.107301 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:19.107269    2583 generic.go:356] "Generic (PLEG): container finished" podID="b20abe23-755e-49f6-a042-2146dd85933e" containerID="9b4a3d231bb410e0c33d44700c28ce6693d3fbe86e0dff17c6362214ee30ec58" exitCode=0
Sep 07 04:04:19.107475 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:19.107312    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"
Sep 07 04:04:19.107475 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:19.107359    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8" event={"ID":"b20abe23-755e-49f6-a042-2146dd85933e","Type":"ContainerDied","Data":"9b4a3d231bb410e0c33d44700c28ce6693d3fbe86e0dff17c6362214ee30ec58"}
Sep 07 04:04:19.107475 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:19.107404    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8" event={"ID":"b20abe23-755e-49f6-a042-2146dd85933e","Type":"ContainerDied","Data":"4868e7a98bd68c687b935df641dc4b9e691343be7cb7d7bc9540d44df2dfdf36"}
Sep 07 04:04:19.107475 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:19.107420    2583 scope.go:117] "RemoveContainer" containerID="9b4a3d231bb410e0c33d44700c28ce6693d3fbe86e0dff17c6362214ee30ec58"
Sep 07 04:04:19.116402 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:19.116386    2583 scope.go:117] "RemoveContainer" containerID="9b4a3d231bb410e0c33d44700c28ce6693d3fbe86e0dff17c6362214ee30ec58"
Sep 07 04:04:19.116660 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:04:19.116642    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b4a3d231bb410e0c33d44700c28ce6693d3fbe86e0dff17c6362214ee30ec58\": container with ID starting with 9b4a3d231bb410e0c33d44700c28ce6693d3fbe86e0dff17c6362214ee30ec58 not found: ID does not exist" containerID="9b4a3d231bb410e0c33d44700c28ce6693d3fbe86e0dff17c6362214ee30ec58"
Sep 07 04:04:19.116697 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:19.116669    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b4a3d231bb410e0c33d44700c28ce6693d3fbe86e0dff17c6362214ee30ec58"} err="failed to get container status \"9b4a3d231bb410e0c33d44700c28ce6693d3fbe86e0dff17c6362214ee30ec58\": rpc error: code = NotFound desc = could not find container \"9b4a3d231bb410e0c33d44700c28ce6693d3fbe86e0dff17c6362214ee30ec58\": container with ID starting with 9b4a3d231bb410e0c33d44700c28ce6693d3fbe86e0dff17c6362214ee30ec58 not found: ID does not exist"
Sep 07 04:04:19.126957 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:19.126935    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"]
Sep 07 04:04:19.130131 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:19.130109    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8"]
Sep 07 04:04:19.522989 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:19.522954    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b20abe23-755e-49f6-a042-2146dd85933e" path="/var/lib/kubelet/pods/b20abe23-755e-49f6-a042-2146dd85933e/volumes"
Sep 07 04:04:19.862488 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:19.862406    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:20.923832 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:20.923799    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:21.026084 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:21.025988    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:21.151710 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:21.151678    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:21.721506 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:21.721475    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:23.377256 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:23.377186    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-a-1038cd/test-model-1038cd-kserve-7bb94f64cf-wmqp8" podUID="b20abe23-755e-49f6-a042-2146dd85933e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 07 04:04:23.432174 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:04:23.432077    2583 event.go:359] "Server rejected event (will not retry!)" err="namespaces \"ai-tenant-e2e-inf-a-1038cd\" not found" event="&Event{ObjectMeta:{test-model-1038cd-kserve-7bb94f64cf-wmqp8.18d2ee2959356f91  ai-tenant-e2e-inf-a-1038cd    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:ai-tenant-e2e-inf-a-1038cd,Name:test-model-1038cd-kserve-7bb94f64cf-wmqp8,UID:b20abe23-755e-49f6-a042-2146dd85933e,APIVersion:v1,ResourceVersion:29335,FieldPath:spec.containers{main},},Reason:Unhealthy,Message:Readiness probe failed: Get \"https://10.134.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers),Source:EventSource{Component:kubelet,Host:ip-10-0-132-175.ec2.internal,},FirstTimestamp:2026-09-07 04:04:23.377252241 +0000 UTC m=+950.310732951,LastTimestamp:2026-09-07 04:04:23.377252241 +0000 UTC m=+950.310732951,Count:1,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-175.ec2.internal,}"
Sep 07 04:04:25.914339 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:25.914299    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:25.989513 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:25.989472    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:26.560592 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:26.560553    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:31.964710 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:31.964674    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"]
Sep 07 04:04:31.965322 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:31.965014    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx" podUID="4b3fbc08-1c66-42e3-b3c6-bdb34726c317" containerName="maas-api" containerID="cri-o://7e3a71326024e1ae8c084b284805cfb8cba4a65e9b3176620aa3009310c213c0" gracePeriod=30
Sep 07 04:04:31.992956 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:31.992919    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:32.018091 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.018044    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b"]
Sep 07 04:04:32.018331 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.018298    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b" podUID="f664f7ca-4a00-45f0-8017-0858b5479076" containerName="payload-processing" containerID="cri-o://c943d232b8b96fb933bc7530eed85f8d458d60fc40e4b3f703c54ed112e88fe0" gracePeriod=30
Sep 07 04:04:32.064187 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.064154    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:32.164328 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.164290    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:32.178421 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.178389    2583 generic.go:356] "Generic (PLEG): container finished" podID="f664f7ca-4a00-45f0-8017-0858b5479076" containerID="c943d232b8b96fb933bc7530eed85f8d458d60fc40e4b3f703c54ed112e88fe0" exitCode=0
Sep 07 04:04:32.178586 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.178466    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b" event={"ID":"f664f7ca-4a00-45f0-8017-0858b5479076","Type":"ContainerDied","Data":"c943d232b8b96fb933bc7530eed85f8d458d60fc40e4b3f703c54ed112e88fe0"}
Sep 07 04:04:32.244859 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.244766    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:32.325317 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.325279    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b"
Sep 07 04:04:32.419631 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.419426    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88fb5\" (UniqueName: \"kubernetes.io/projected/f664f7ca-4a00-45f0-8017-0858b5479076-kube-api-access-88fb5\") pod \"f664f7ca-4a00-45f0-8017-0858b5479076\" (UID: \"f664f7ca-4a00-45f0-8017-0858b5479076\") "
Sep 07 04:04:32.419631 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.419477    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f664f7ca-4a00-45f0-8017-0858b5479076-plugins-config-volume\") pod \"f664f7ca-4a00-45f0-8017-0858b5479076\" (UID: \"f664f7ca-4a00-45f0-8017-0858b5479076\") "
Sep 07 04:04:32.419631 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.419492    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"
Sep 07 04:04:32.419978 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.419951    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f664f7ca-4a00-45f0-8017-0858b5479076-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f664f7ca-4a00-45f0-8017-0858b5479076" (UID: "f664f7ca-4a00-45f0-8017-0858b5479076"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 04:04:32.422280 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.422252    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f664f7ca-4a00-45f0-8017-0858b5479076-kube-api-access-88fb5" (OuterVolumeSpecName: "kube-api-access-88fb5") pod "f664f7ca-4a00-45f0-8017-0858b5479076" (UID: "f664f7ca-4a00-45f0-8017-0858b5479076"). InnerVolumeSpecName "kube-api-access-88fb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:04:32.518246 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.518153    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:32.520261 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.520237    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-maas-api-tls\") pod \"4b3fbc08-1c66-42e3-b3c6-bdb34726c317\" (UID: \"4b3fbc08-1c66-42e3-b3c6-bdb34726c317\") "
Sep 07 04:04:32.520336 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.520322    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-openshift-service-ca\") pod \"4b3fbc08-1c66-42e3-b3c6-bdb34726c317\" (UID: \"4b3fbc08-1c66-42e3-b3c6-bdb34726c317\") "
Sep 07 04:04:32.520374 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.520355    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hprw2\" (UniqueName: \"kubernetes.io/projected/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-kube-api-access-hprw2\") pod \"4b3fbc08-1c66-42e3-b3c6-bdb34726c317\" (UID: \"4b3fbc08-1c66-42e3-b3c6-bdb34726c317\") "
Sep 07 04:04:32.520419 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.520403    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-metrics-certs\") pod \"4b3fbc08-1c66-42e3-b3c6-bdb34726c317\" (UID: \"4b3fbc08-1c66-42e3-b3c6-bdb34726c317\") "
Sep 07 04:04:32.520700 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.520680    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88fb5\" (UniqueName: \"kubernetes.io/projected/f664f7ca-4a00-45f0-8017-0858b5479076-kube-api-access-88fb5\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:04:32.520800 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.520709    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f664f7ca-4a00-45f0-8017-0858b5479076-plugins-config-volume\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:04:32.520800 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.520677    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4b3fbc08-1c66-42e3-b3c6-bdb34726c317" (UID: "4b3fbc08-1c66-42e3-b3c6-bdb34726c317"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 04:04:32.522615 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.522586    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4b3fbc08-1c66-42e3-b3c6-bdb34726c317" (UID: "4b3fbc08-1c66-42e3-b3c6-bdb34726c317"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:04:32.522999 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.522982    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-kube-api-access-hprw2" (OuterVolumeSpecName: "kube-api-access-hprw2") pod "4b3fbc08-1c66-42e3-b3c6-bdb34726c317" (UID: "4b3fbc08-1c66-42e3-b3c6-bdb34726c317"). InnerVolumeSpecName "kube-api-access-hprw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:04:32.523056 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.523000    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4b3fbc08-1c66-42e3-b3c6-bdb34726c317" (UID: "4b3fbc08-1c66-42e3-b3c6-bdb34726c317"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:04:32.621298 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.621259    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-maas-api-tls\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:04:32.621298 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.621292    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-openshift-service-ca\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:04:32.621563 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.621307    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hprw2\" (UniqueName: \"kubernetes.io/projected/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-kube-api-access-hprw2\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:04:32.621563 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.621321    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b3fbc08-1c66-42e3-b3c6-bdb34726c317-metrics-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:04:32.809650 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.809559    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:32.911435 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:32.911402    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:33.043110 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:33.043076    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:33.184900 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:33.184877    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b"
Sep 07 04:04:33.184900 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:33.184884    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b" event={"ID":"f664f7ca-4a00-45f0-8017-0858b5479076","Type":"ContainerDied","Data":"1061cff5b2a0b0d0a7890ea78bfac20aec0d32c03867f83d24b7752be765cd53"}
Sep 07 04:04:33.185154 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:33.184962    2583 scope.go:117] "RemoveContainer" containerID="c943d232b8b96fb933bc7530eed85f8d458d60fc40e4b3f703c54ed112e88fe0"
Sep 07 04:04:33.186530 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:33.186485    2583 generic.go:356] "Generic (PLEG): container finished" podID="4b3fbc08-1c66-42e3-b3c6-bdb34726c317" containerID="7e3a71326024e1ae8c084b284805cfb8cba4a65e9b3176620aa3009310c213c0" exitCode=0
Sep 07 04:04:33.186787 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:33.186534    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"
Sep 07 04:04:33.186787 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:33.186558    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx" event={"ID":"4b3fbc08-1c66-42e3-b3c6-bdb34726c317","Type":"ContainerDied","Data":"7e3a71326024e1ae8c084b284805cfb8cba4a65e9b3176620aa3009310c213c0"}
Sep 07 04:04:33.186787 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:33.186597    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx" event={"ID":"4b3fbc08-1c66-42e3-b3c6-bdb34726c317","Type":"ContainerDied","Data":"97ed250cff6f17025532ef805f1fea350b21fbea6e6ef126a47fad2991730a77"}
Sep 07 04:04:33.195482 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:33.195463    2583 scope.go:117] "RemoveContainer" containerID="7e3a71326024e1ae8c084b284805cfb8cba4a65e9b3176620aa3009310c213c0"
Sep 07 04:04:33.204095 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:33.204075    2583 scope.go:117] "RemoveContainer" containerID="7e3a71326024e1ae8c084b284805cfb8cba4a65e9b3176620aa3009310c213c0"
Sep 07 04:04:33.204506 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:04:33.204466    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e3a71326024e1ae8c084b284805cfb8cba4a65e9b3176620aa3009310c213c0\": container with ID starting with 7e3a71326024e1ae8c084b284805cfb8cba4a65e9b3176620aa3009310c213c0 not found: ID does not exist" containerID="7e3a71326024e1ae8c084b284805cfb8cba4a65e9b3176620aa3009310c213c0"
Sep 07 04:04:33.204626 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:33.204511    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e3a71326024e1ae8c084b284805cfb8cba4a65e9b3176620aa3009310c213c0"} err="failed to get container status \"7e3a71326024e1ae8c084b284805cfb8cba4a65e9b3176620aa3009310c213c0\": rpc error: code = NotFound desc = could not find container \"7e3a71326024e1ae8c084b284805cfb8cba4a65e9b3176620aa3009310c213c0\": container with ID starting with 7e3a71326024e1ae8c084b284805cfb8cba4a65e9b3176620aa3009310c213c0 not found: ID does not exist"
Sep 07 04:04:33.206759 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:33.206738    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b"]
Sep 07 04:04:33.210877 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:33.210859    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a8da4e-7b5774948f-6957b"]
Sep 07 04:04:33.225240 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:33.225202    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"]
Sep 07 04:04:33.231764 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:33.231744    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a8da4e-64d7b5b55-vz9vx"]
Sep 07 04:04:33.358450 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:33.358415    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:33.523167 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:33.523079    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b3fbc08-1c66-42e3-b3c6-bdb34726c317" path="/var/lib/kubelet/pods/4b3fbc08-1c66-42e3-b3c6-bdb34726c317/volumes"
Sep 07 04:04:33.523527 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:33.523511    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f664f7ca-4a00-45f0-8017-0858b5479076" path="/var/lib/kubelet/pods/f664f7ca-4a00-45f0-8017-0858b5479076/volumes"
Sep 07 04:04:33.936990 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:33.936954    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:36.499780 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:36.499743    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:36.592079 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:36.592048    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:37.598157 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.598128    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"]
Sep 07 04:04:37.598690 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.598661    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b3fbc08-1c66-42e3-b3c6-bdb34726c317" containerName="maas-api"
Sep 07 04:04:37.598690 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.598688    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b3fbc08-1c66-42e3-b3c6-bdb34726c317" containerName="maas-api"
Sep 07 04:04:37.598885 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.598707    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f664f7ca-4a00-45f0-8017-0858b5479076" containerName="payload-processing"
Sep 07 04:04:37.598885 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.598716    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f664f7ca-4a00-45f0-8017-0858b5479076" containerName="payload-processing"
Sep 07 04:04:37.598885 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.598740    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3" containerName="maas-api"
Sep 07 04:04:37.598885 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.598749    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3" containerName="maas-api"
Sep 07 04:04:37.598885 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.598765    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9375483-9747-43fa-aba4-6e81fe6e87d3" containerName="payload-processing"
Sep 07 04:04:37.598885 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.598774    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9375483-9747-43fa-aba4-6e81fe6e87d3" containerName="payload-processing"
Sep 07 04:04:37.598885 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.598785    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b20abe23-755e-49f6-a042-2146dd85933e" containerName="main"
Sep 07 04:04:37.598885 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.598794    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b20abe23-755e-49f6-a042-2146dd85933e" containerName="main"
Sep 07 04:04:37.598885 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.598813    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9b44b17-73cc-4c87-8d64-8bcf7bb10f24" containerName="maas-api"
Sep 07 04:04:37.598885 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.598824    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9b44b17-73cc-4c87-8d64-8bcf7bb10f24" containerName="maas-api"
Sep 07 04:04:37.599270 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.598923    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9375483-9747-43fa-aba4-6e81fe6e87d3" containerName="payload-processing"
Sep 07 04:04:37.599270 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.598939    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b3fbc08-1c66-42e3-b3c6-bdb34726c317" containerName="maas-api"
Sep 07 04:04:37.599270 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.598948    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fa04728-985e-40e6-b4b5-0c3fb8e4f8a3" containerName="maas-api"
Sep 07 04:04:37.599270 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.598955    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9b44b17-73cc-4c87-8d64-8bcf7bb10f24" containerName="maas-api"
Sep 07 04:04:37.599270 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.598964    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f664f7ca-4a00-45f0-8017-0858b5479076" containerName="payload-processing"
Sep 07 04:04:37.599270 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.598970    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b20abe23-755e-49f6-a042-2146dd85933e" containerName="main"
Sep 07 04:04:37.603671 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.603649    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"
Sep 07 04:04:37.605902 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.605884    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-a5a838b2-serving-cert\""
Sep 07 04:04:37.610813 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.610745    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"]
Sep 07 04:04:37.633235 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.633190    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:37.733290 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.733255    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:37.769115 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.769087    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9da6b53-af0c-456d-adbb-3a68bf998db3-maas-api-tls\") pod \"maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj\" (UID: \"e9da6b53-af0c-456d-adbb-3a68bf998db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"
Sep 07 04:04:37.769115 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.769116    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9da6b53-af0c-456d-adbb-3a68bf998db3-openshift-service-ca\") pod \"maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj\" (UID: \"e9da6b53-af0c-456d-adbb-3a68bf998db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"
Sep 07 04:04:37.769333 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.769151    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj5dm\" (UniqueName: \"kubernetes.io/projected/e9da6b53-af0c-456d-adbb-3a68bf998db3-kube-api-access-wj5dm\") pod \"maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj\" (UID: \"e9da6b53-af0c-456d-adbb-3a68bf998db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"
Sep 07 04:04:37.769333 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.769170    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9da6b53-af0c-456d-adbb-3a68bf998db3-metrics-certs\") pod \"maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj\" (UID: \"e9da6b53-af0c-456d-adbb-3a68bf998db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"
Sep 07 04:04:37.870378 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.870286    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9da6b53-af0c-456d-adbb-3a68bf998db3-maas-api-tls\") pod \"maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj\" (UID: \"e9da6b53-af0c-456d-adbb-3a68bf998db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"
Sep 07 04:04:37.870378 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.870322    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9da6b53-af0c-456d-adbb-3a68bf998db3-openshift-service-ca\") pod \"maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj\" (UID: \"e9da6b53-af0c-456d-adbb-3a68bf998db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"
Sep 07 04:04:37.870378 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.870369    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj5dm\" (UniqueName: \"kubernetes.io/projected/e9da6b53-af0c-456d-adbb-3a68bf998db3-kube-api-access-wj5dm\") pod \"maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj\" (UID: \"e9da6b53-af0c-456d-adbb-3a68bf998db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"
Sep 07 04:04:37.870654 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.870479    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9da6b53-af0c-456d-adbb-3a68bf998db3-metrics-certs\") pod \"maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj\" (UID: \"e9da6b53-af0c-456d-adbb-3a68bf998db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"
Sep 07 04:04:37.870983 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.870962    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9da6b53-af0c-456d-adbb-3a68bf998db3-openshift-service-ca\") pod \"maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj\" (UID: \"e9da6b53-af0c-456d-adbb-3a68bf998db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"
Sep 07 04:04:37.872894 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.872872    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9da6b53-af0c-456d-adbb-3a68bf998db3-metrics-certs\") pod \"maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj\" (UID: \"e9da6b53-af0c-456d-adbb-3a68bf998db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"
Sep 07 04:04:37.873066 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.873044    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9da6b53-af0c-456d-adbb-3a68bf998db3-maas-api-tls\") pod \"maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj\" (UID: \"e9da6b53-af0c-456d-adbb-3a68bf998db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"
Sep 07 04:04:37.878495 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.878469    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj5dm\" (UniqueName: \"kubernetes.io/projected/e9da6b53-af0c-456d-adbb-3a68bf998db3-kube-api-access-wj5dm\") pod \"maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj\" (UID: \"e9da6b53-af0c-456d-adbb-3a68bf998db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"
Sep 07 04:04:37.915446 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:37.915407    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"
Sep 07 04:04:38.198906 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:38.198877    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"]
Sep 07 04:04:38.201454 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:04:38.201422    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9da6b53_af0c_456d_adbb_3a68bf998db3.slice/crio-70312801e273ceaf9a501d8a4bc851e76665e88904e8144be1d832c89aee3a28 WatchSource:0}: Error finding container 70312801e273ceaf9a501d8a4bc851e76665e88904e8144be1d832c89aee3a28: Status 404 returned error can't find the container with id 70312801e273ceaf9a501d8a4bc851e76665e88904e8144be1d832c89aee3a28
Sep 07 04:04:38.205590 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:38.205562    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:38.210053 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:38.210031    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj" event={"ID":"e9da6b53-af0c-456d-adbb-3a68bf998db3","Type":"ContainerStarted","Data":"70312801e273ceaf9a501d8a4bc851e76665e88904e8144be1d832c89aee3a28"}
Sep 07 04:04:38.315423 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:38.315387    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:39.216861 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:39.216825    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj" event={"ID":"e9da6b53-af0c-456d-adbb-3a68bf998db3","Type":"ContainerStarted","Data":"ac55222547d7b03fdfd531c8587fcf9b901f0522540d704b09d0b3665b18a2de"}
Sep 07 04:04:39.217292 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:39.216886    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"
Sep 07 04:04:39.249902 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:39.249846    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj" podStartSLOduration=1.780531015 podStartE2EDuration="2.249829899s" totalImagesPullingTime="469.298884ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:04:39.247712953 +0000 UTC m=+966.181193671" watchObservedRunningTime="2026-09-07 04:04:39.249829899 +0000 UTC m=+966.183310966"
Sep 07 04:04:39.281395 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:39.281361    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:39.342318 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:39.342289    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:39.544138 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:39.544063    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:40.077804 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:40.077773    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:40.391678 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:40.391598    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:40.491869 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:40.491840    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:43.043567 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:43.043531    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:43.137440 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:43.137408    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:44.142128 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.142093    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"]
Sep 07 04:04:44.146004 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.145979    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"
Sep 07 04:04:44.148254 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.148207    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-b8cea6-serving-cert\""
Sep 07 04:04:44.155828 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.155730    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"]
Sep 07 04:04:44.224669 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.224638    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft\" (UID: \"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"
Sep 07 04:04:44.224845 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.224683    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft\" (UID: \"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"
Sep 07 04:04:44.224845 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.224785    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9jds\" (UniqueName: \"kubernetes.io/projected/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-kube-api-access-b9jds\") pod \"maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft\" (UID: \"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"
Sep 07 04:04:44.224845 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.224832    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft\" (UID: \"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"
Sep 07 04:04:44.227901 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.227871    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:44.326823 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.326792    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft\" (UID: \"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"
Sep 07 04:04:44.327035 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.326845    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft\" (UID: \"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"
Sep 07 04:04:44.327035 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.326926    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9jds\" (UniqueName: \"kubernetes.io/projected/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-kube-api-access-b9jds\") pod \"maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft\" (UID: \"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"
Sep 07 04:04:44.327035 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.326981    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft\" (UID: \"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"
Sep 07 04:04:44.327702 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.327617    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft\" (UID: \"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"
Sep 07 04:04:44.329523 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.329497    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft\" (UID: \"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"
Sep 07 04:04:44.329640 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.329620    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft\" (UID: \"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"
Sep 07 04:04:44.330971 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.330948    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:44.334713 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.334695    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9jds\" (UniqueName: \"kubernetes.io/projected/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-kube-api-access-b9jds\") pod \"maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft\" (UID: \"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"
Sep 07 04:04:44.431169 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.431085    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:44.461309 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.461276    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"
Sep 07 04:04:44.707236 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.703703    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:44.735438 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.735388    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"]
Sep 07 04:04:44.737973 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:04:44.737945    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ae9093c_0ed4_476d_abde_ccaed0cf2ad8.slice/crio-77494320a49bff0e04c7a5e2b2a3968bf01de6ba5d5ed4cc14fb55505a800bf7 WatchSource:0}: Error finding container 77494320a49bff0e04c7a5e2b2a3968bf01de6ba5d5ed4cc14fb55505a800bf7: Status 404 returned error can't find the container with id 77494320a49bff0e04c7a5e2b2a3968bf01de6ba5d5ed4cc14fb55505a800bf7
Sep 07 04:04:44.802510 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.802474    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:44.833021 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.832989    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74"]
Sep 07 04:04:44.837673 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.837651    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74"
Sep 07 04:04:44.840152 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.840126    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-b8cea6\""
Sep 07 04:04:44.840293 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.840230    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-b8cea6-dockercfg-b2w4k\""
Sep 07 04:04:44.847139 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.847113    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74"]
Sep 07 04:04:44.931745 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.931713    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfql9\" (UniqueName: \"kubernetes.io/projected/80646e0f-ef88-407d-b373-2e84d7517194-kube-api-access-bfql9\") pod \"payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74\" (UID: \"80646e0f-ef88-407d-b373-2e84d7517194\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74"
Sep 07 04:04:44.931953 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:44.931774    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80646e0f-ef88-407d-b373-2e84d7517194-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74\" (UID: \"80646e0f-ef88-407d-b373-2e84d7517194\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74"
Sep 07 04:04:45.017269 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:45.017168    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:45.032586 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:45.032554    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfql9\" (UniqueName: \"kubernetes.io/projected/80646e0f-ef88-407d-b373-2e84d7517194-kube-api-access-bfql9\") pod \"payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74\" (UID: \"80646e0f-ef88-407d-b373-2e84d7517194\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74"
Sep 07 04:04:45.032775 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:45.032618    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80646e0f-ef88-407d-b373-2e84d7517194-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74\" (UID: \"80646e0f-ef88-407d-b373-2e84d7517194\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74"
Sep 07 04:04:45.033317 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:45.033297    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80646e0f-ef88-407d-b373-2e84d7517194-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74\" (UID: \"80646e0f-ef88-407d-b373-2e84d7517194\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74"
Sep 07 04:04:45.041325 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:45.041294    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfql9\" (UniqueName: \"kubernetes.io/projected/80646e0f-ef88-407d-b373-2e84d7517194-kube-api-access-bfql9\") pod \"payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74\" (UID: \"80646e0f-ef88-407d-b373-2e84d7517194\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74"
Sep 07 04:04:45.149362 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:45.149314    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74"
Sep 07 04:04:45.228331 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:45.228298    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"
Sep 07 04:04:45.248860 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:45.248805    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft" event={"ID":"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8","Type":"ContainerStarted","Data":"77494320a49bff0e04c7a5e2b2a3968bf01de6ba5d5ed4cc14fb55505a800bf7"}
Sep 07 04:04:45.425147 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:45.425122    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74"]
Sep 07 04:04:45.427527 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:04:45.427499    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80646e0f_ef88_407d_b373_2e84d7517194.slice/crio-8e6e178729eb58207eff752894d475c957a07b5722b28eb7742d84b79e3dd224 WatchSource:0}: Error finding container 8e6e178729eb58207eff752894d475c957a07b5722b28eb7742d84b79e3dd224: Status 404 returned error can't find the container with id 8e6e178729eb58207eff752894d475c957a07b5722b28eb7742d84b79e3dd224
Sep 07 04:04:46.257375 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:46.257337    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74" event={"ID":"80646e0f-ef88-407d-b373-2e84d7517194","Type":"ContainerStarted","Data":"460d83d363b57c0aa21e87151c60f611b88473fc604af7aa19267380c82089b4"}
Sep 07 04:04:46.257818 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:46.257404    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74" event={"ID":"80646e0f-ef88-407d-b373-2e84d7517194","Type":"ContainerStarted","Data":"8e6e178729eb58207eff752894d475c957a07b5722b28eb7742d84b79e3dd224"}
Sep 07 04:04:46.257818 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:46.257462    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74"
Sep 07 04:04:46.259026 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:46.259003    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft" event={"ID":"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8","Type":"ContainerStarted","Data":"c5f509b8804b7edd6f7ddf22a52bff2e032e2843412d76fbba6c3cfaf32e45a8"}
Sep 07 04:04:46.259248 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:46.259201    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"
Sep 07 04:04:46.272667 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:46.272614    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74" podStartSLOduration=2.272601767 podStartE2EDuration="2.272601767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:04:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:04:46.271906887 +0000 UTC m=+973.205387606" watchObservedRunningTime="2026-09-07 04:04:46.272601767 +0000 UTC m=+973.206082485"
Sep 07 04:04:46.288538 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:46.288492    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft" podStartSLOduration=1.75668166 podStartE2EDuration="2.288475681s" totalImagesPullingTime="531.794021ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:04:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:04:46.287004547 +0000 UTC m=+973.220485269" watchObservedRunningTime="2026-09-07 04:04:46.288475681 +0000 UTC m=+973.221956430"
Sep 07 04:04:48.398200 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:48.398169    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:52.269050 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:52.269019    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"
Sep 07 04:04:53.678325 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:53.678296    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:04:57.266399 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:57.266315    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74"
Sep 07 04:04:59.512798 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:04:59.512759    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:01.459125 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:01.459093    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:02.115828 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:02.115793    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:02.218123 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:02.218095    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:02.545159 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:02.545121    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:05.058176 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:05.058139    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:07.886536 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:07.886502    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:08.199492 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:08.199458    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:08.304280 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:08.304234    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:08.358647 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:08.358619    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:08.490389 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:08.490301    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:08.840603 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:08.840511    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:08.940779 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:08.940745    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:11.718619 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:11.718581    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:11.818632 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:11.818590    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:12.880570 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:12.880540    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:12.981018 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:12.980986    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:13.340492 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:13.340459    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:13.429430 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:13.429396    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:18.790513 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:18.790469    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:19.253659 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:19.253625    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"]
Sep 07 04:05:19.253892 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:19.253866    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj" podUID="e9da6b53-af0c-456d-adbb-3a68bf998db3" containerName="maas-api" containerID="cri-o://ac55222547d7b03fdfd531c8587fcf9b901f0522540d704b09d0b3665b18a2de" gracePeriod=30
Sep 07 04:05:19.280667 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:19.280634    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:19.354776 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:19.354693    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:19.421726 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:19.421691    2583 generic.go:356] "Generic (PLEG): container finished" podID="e9da6b53-af0c-456d-adbb-3a68bf998db3" containerID="ac55222547d7b03fdfd531c8587fcf9b901f0522540d704b09d0b3665b18a2de" exitCode=0
Sep 07 04:05:19.421890 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:19.421856    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj" event={"ID":"e9da6b53-af0c-456d-adbb-3a68bf998db3","Type":"ContainerDied","Data":"ac55222547d7b03fdfd531c8587fcf9b901f0522540d704b09d0b3665b18a2de"}
Sep 07 04:05:19.452386 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:19.452350    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:19.508005 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:19.507925    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:19.515442 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:19.515416    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"
Sep 07 04:05:19.671372 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:19.671337    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9da6b53-af0c-456d-adbb-3a68bf998db3-openshift-service-ca\") pod \"e9da6b53-af0c-456d-adbb-3a68bf998db3\" (UID: \"e9da6b53-af0c-456d-adbb-3a68bf998db3\") "
Sep 07 04:05:19.671567 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:19.671414    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9da6b53-af0c-456d-adbb-3a68bf998db3-maas-api-tls\") pod \"e9da6b53-af0c-456d-adbb-3a68bf998db3\" (UID: \"e9da6b53-af0c-456d-adbb-3a68bf998db3\") "
Sep 07 04:05:19.671567 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:19.671449    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9da6b53-af0c-456d-adbb-3a68bf998db3-metrics-certs\") pod \"e9da6b53-af0c-456d-adbb-3a68bf998db3\" (UID: \"e9da6b53-af0c-456d-adbb-3a68bf998db3\") "
Sep 07 04:05:19.671567 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:19.671539    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wj5dm\" (UniqueName: \"kubernetes.io/projected/e9da6b53-af0c-456d-adbb-3a68bf998db3-kube-api-access-wj5dm\") pod \"e9da6b53-af0c-456d-adbb-3a68bf998db3\" (UID: \"e9da6b53-af0c-456d-adbb-3a68bf998db3\") "
Sep 07 04:05:19.671744 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:19.671717    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9da6b53-af0c-456d-adbb-3a68bf998db3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e9da6b53-af0c-456d-adbb-3a68bf998db3" (UID: "e9da6b53-af0c-456d-adbb-3a68bf998db3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 04:05:19.673901 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:19.673863    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9da6b53-af0c-456d-adbb-3a68bf998db3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e9da6b53-af0c-456d-adbb-3a68bf998db3" (UID: "e9da6b53-af0c-456d-adbb-3a68bf998db3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:05:19.673901 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:19.673893    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9da6b53-af0c-456d-adbb-3a68bf998db3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e9da6b53-af0c-456d-adbb-3a68bf998db3" (UID: "e9da6b53-af0c-456d-adbb-3a68bf998db3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:05:19.674079 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:19.673907    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9da6b53-af0c-456d-adbb-3a68bf998db3-kube-api-access-wj5dm" (OuterVolumeSpecName: "kube-api-access-wj5dm") pod "e9da6b53-af0c-456d-adbb-3a68bf998db3" (UID: "e9da6b53-af0c-456d-adbb-3a68bf998db3"). InnerVolumeSpecName "kube-api-access-wj5dm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:05:19.772400 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:19.772320    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wj5dm\" (UniqueName: \"kubernetes.io/projected/e9da6b53-af0c-456d-adbb-3a68bf998db3-kube-api-access-wj5dm\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:05:19.772400 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:19.772344    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9da6b53-af0c-456d-adbb-3a68bf998db3-openshift-service-ca\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:05:19.772400 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:19.772356    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9da6b53-af0c-456d-adbb-3a68bf998db3-maas-api-tls\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:05:19.772400 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:19.772370    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9da6b53-af0c-456d-adbb-3a68bf998db3-metrics-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:05:19.772768 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:19.772750    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:19.872926 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:19.872893    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:20.428840 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:20.428810    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"
Sep 07 04:05:20.428840 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:20.428816    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj" event={"ID":"e9da6b53-af0c-456d-adbb-3a68bf998db3","Type":"ContainerDied","Data":"70312801e273ceaf9a501d8a4bc851e76665e88904e8144be1d832c89aee3a28"}
Sep 07 04:05:20.429101 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:20.428883    2583 scope.go:117] "RemoveContainer" containerID="ac55222547d7b03fdfd531c8587fcf9b901f0522540d704b09d0b3665b18a2de"
Sep 07 04:05:20.450390 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:20.450362    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"]
Sep 07 04:05:20.453771 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:20.453746    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a5a838b2-745bfbcf7f-5c7cj"]
Sep 07 04:05:21.522001 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:21.521968    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9da6b53-af0c-456d-adbb-3a68bf998db3" path="/var/lib/kubelet/pods/e9da6b53-af0c-456d-adbb-3a68bf998db3/volumes"
Sep 07 04:05:25.644582 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:25.644540    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:25.698846 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:25.698809    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:25.902583 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:25.902499    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:26.231795 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:26.231761    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:26.333255 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:26.333205    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:26.459617 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:26.459582    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:32.075397 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:32.075359    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:32.183414 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:32.183376    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:32.275603 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:32.275572    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:32.434439 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:32.434408    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:33.812502 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:33.812466    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:36.686872 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:36.686820    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:38.074928 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:38.074895    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:38.175783 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:38.175749    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:38.956364 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:38.956325    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:39.240874 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:39.240805    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:39.761314 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:39.761265    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:41.957366 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:41.957328    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:42.057370 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:42.057331    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:43.067628 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.067585    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"]
Sep 07 04:05:43.068076 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.068025    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9da6b53-af0c-456d-adbb-3a68bf998db3" containerName="maas-api"
Sep 07 04:05:43.068076 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.068047    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9da6b53-af0c-456d-adbb-3a68bf998db3" containerName="maas-api"
Sep 07 04:05:43.068328 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.068148    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9da6b53-af0c-456d-adbb-3a68bf998db3" containerName="maas-api"
Sep 07 04:05:43.072496 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.072474    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"
Sep 07 04:05:43.074744 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.074725    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-747fee-serving-cert\""
Sep 07 04:05:43.083092 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.083064    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"]
Sep 07 04:05:43.100822 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.100791    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:43.178903 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.178867    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fdee0a9-1427-4f4a-897d-594ea81e87b9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh\" (UID: \"3fdee0a9-1427-4f4a-897d-594ea81e87b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"
Sep 07 04:05:43.178903 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.178907    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fdee0a9-1427-4f4a-897d-594ea81e87b9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh\" (UID: \"3fdee0a9-1427-4f4a-897d-594ea81e87b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"
Sep 07 04:05:43.179157 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.178931    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fdee0a9-1427-4f4a-897d-594ea81e87b9-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh\" (UID: \"3fdee0a9-1427-4f4a-897d-594ea81e87b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"
Sep 07 04:05:43.179157 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.179024    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcckq\" (UniqueName: \"kubernetes.io/projected/3fdee0a9-1427-4f4a-897d-594ea81e87b9-kube-api-access-tcckq\") pod \"maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh\" (UID: \"3fdee0a9-1427-4f4a-897d-594ea81e87b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"
Sep 07 04:05:43.200898 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.200861    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:43.280293 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.280260    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fdee0a9-1427-4f4a-897d-594ea81e87b9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh\" (UID: \"3fdee0a9-1427-4f4a-897d-594ea81e87b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"
Sep 07 04:05:43.280465 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.280302    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fdee0a9-1427-4f4a-897d-594ea81e87b9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh\" (UID: \"3fdee0a9-1427-4f4a-897d-594ea81e87b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"
Sep 07 04:05:43.280465 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.280322    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fdee0a9-1427-4f4a-897d-594ea81e87b9-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh\" (UID: \"3fdee0a9-1427-4f4a-897d-594ea81e87b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"
Sep 07 04:05:43.280465 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.280370    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcckq\" (UniqueName: \"kubernetes.io/projected/3fdee0a9-1427-4f4a-897d-594ea81e87b9-kube-api-access-tcckq\") pod \"maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh\" (UID: \"3fdee0a9-1427-4f4a-897d-594ea81e87b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"
Sep 07 04:05:43.280465 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:05:43.280426    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-serving-cert: secret "maas-api-e2e-shared-a-w2-747fee-serving-cert" not found
Sep 07 04:05:43.280692 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:05:43.280496    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fdee0a9-1427-4f4a-897d-594ea81e87b9-maas-api-tls podName:3fdee0a9-1427-4f4a-897d-594ea81e87b9 nodeName:}" failed. No retries permitted until 2026-09-07 04:05:43.780478489 +0000 UTC m=+1030.713959189 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3fdee0a9-1427-4f4a-897d-594ea81e87b9-maas-api-tls") pod "maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh" (UID: "3fdee0a9-1427-4f4a-897d-594ea81e87b9") : secret "maas-api-e2e-shared-a-w2-747fee-serving-cert" not found
Sep 07 04:05:43.281021 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.280999    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fdee0a9-1427-4f4a-897d-594ea81e87b9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh\" (UID: \"3fdee0a9-1427-4f4a-897d-594ea81e87b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"
Sep 07 04:05:43.283003 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.282985    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fdee0a9-1427-4f4a-897d-594ea81e87b9-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh\" (UID: \"3fdee0a9-1427-4f4a-897d-594ea81e87b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"
Sep 07 04:05:43.288460 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.288440    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcckq\" (UniqueName: \"kubernetes.io/projected/3fdee0a9-1427-4f4a-897d-594ea81e87b9-kube-api-access-tcckq\") pod \"maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh\" (UID: \"3fdee0a9-1427-4f4a-897d-594ea81e87b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"
Sep 07 04:05:43.483158 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.483118    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:43.583305 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.583273    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:43.653971 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.653943    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:43.785932 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.785832    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fdee0a9-1427-4f4a-897d-594ea81e87b9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh\" (UID: \"3fdee0a9-1427-4f4a-897d-594ea81e87b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"
Sep 07 04:05:43.788617 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.788591    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fdee0a9-1427-4f4a-897d-594ea81e87b9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh\" (UID: \"3fdee0a9-1427-4f4a-897d-594ea81e87b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"
Sep 07 04:05:43.987636 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:43.987600    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"
Sep 07 04:05:44.263839 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:44.263500    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"]
Sep 07 04:05:44.266865 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:05:44.266840    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fdee0a9_1427_4f4a_897d_594ea81e87b9.slice/crio-f1a0bc06baf5c3a91be8646b12b51a13e01816d02bd484903a12e1ecfce9671e WatchSource:0}: Error finding container f1a0bc06baf5c3a91be8646b12b51a13e01816d02bd484903a12e1ecfce9671e: Status 404 returned error can't find the container with id f1a0bc06baf5c3a91be8646b12b51a13e01816d02bd484903a12e1ecfce9671e
Sep 07 04:05:44.269485 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:44.268872    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 04:05:44.553979 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:44.553948    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh" event={"ID":"3fdee0a9-1427-4f4a-897d-594ea81e87b9","Type":"ContainerStarted","Data":"f1a0bc06baf5c3a91be8646b12b51a13e01816d02bd484903a12e1ecfce9671e"}
Sep 07 04:05:45.560226 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:45.560174    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh" event={"ID":"3fdee0a9-1427-4f4a-897d-594ea81e87b9","Type":"ContainerStarted","Data":"86e5fcb303ea755c9a3776bd7bcea7d724639f2e875f9e6fc7c1963ba8337acb"}
Sep 07 04:05:45.560617 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:45.560245    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"
Sep 07 04:05:45.577650 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:45.577604    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh" podStartSLOduration=2.159716094 podStartE2EDuration="2.577589184s" totalImagesPullingTime="417.87309ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:05:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:05:45.575383016 +0000 UTC m=+1032.508863813" watchObservedRunningTime="2026-09-07 04:05:45.577589184 +0000 UTC m=+1032.511069902"
Sep 07 04:05:51.571191 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:51.571157    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"
Sep 07 04:05:56.269576 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:56.269538    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:05:56.370114 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:05:56.370077    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:06.908576 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:06.908547    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:07.235304 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:07.235270    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:07.531335 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:07.531245    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:07.916476 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:07.916362    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:10.170687 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:10.170645    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:10.270472 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:10.270427    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:11.399263 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:11.399228    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:11.489107 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:11.489064    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:11.607364 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:11.607329    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:11.954429 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:11.954400    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:12.029671 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.029638    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:12.034616 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.034588    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9"]
Sep 07 04:06:12.038101 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.038080    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9"
Sep 07 04:06:12.040664 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.040646    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-a89024-dockercfg-dnhxl\""
Sep 07 04:06:12.040909 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.040894    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-a89024\""
Sep 07 04:06:12.048904 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.048881    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9"]
Sep 07 04:06:12.052038 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.052016    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4d9c33f-f46c-40e9-887b-b4a303669bdb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9\" (UID: \"f4d9c33f-f46c-40e9-887b-b4a303669bdb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9"
Sep 07 04:06:12.052141 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.052073    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjgsf\" (UniqueName: \"kubernetes.io/projected/f4d9c33f-f46c-40e9-887b-b4a303669bdb-kube-api-access-bjgsf\") pod \"payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9\" (UID: \"f4d9c33f-f46c-40e9-887b-b4a303669bdb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9"
Sep 07 04:06:12.082656 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.082628    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t"]
Sep 07 04:06:12.086067 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.086049    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t"
Sep 07 04:06:12.096991 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.096965    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t"]
Sep 07 04:06:12.153608 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.153577    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjgsf\" (UniqueName: \"kubernetes.io/projected/f4d9c33f-f46c-40e9-887b-b4a303669bdb-kube-api-access-bjgsf\") pod \"payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9\" (UID: \"f4d9c33f-f46c-40e9-887b-b4a303669bdb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9"
Sep 07 04:06:12.153789 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.153616    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6103b3e4-bc89-4ccb-858b-0ba09f16aba1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t\" (UID: \"6103b3e4-bc89-4ccb-858b-0ba09f16aba1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t"
Sep 07 04:06:12.153789 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.153665    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8dbf\" (UniqueName: \"kubernetes.io/projected/6103b3e4-bc89-4ccb-858b-0ba09f16aba1-kube-api-access-w8dbf\") pod \"payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t\" (UID: \"6103b3e4-bc89-4ccb-858b-0ba09f16aba1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t"
Sep 07 04:06:12.153789 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.153694    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4d9c33f-f46c-40e9-887b-b4a303669bdb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9\" (UID: \"f4d9c33f-f46c-40e9-887b-b4a303669bdb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9"
Sep 07 04:06:12.154443 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.154418    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4d9c33f-f46c-40e9-887b-b4a303669bdb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9\" (UID: \"f4d9c33f-f46c-40e9-887b-b4a303669bdb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9"
Sep 07 04:06:12.164546 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.164521    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjgsf\" (UniqueName: \"kubernetes.io/projected/f4d9c33f-f46c-40e9-887b-b4a303669bdb-kube-api-access-bjgsf\") pod \"payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9\" (UID: \"f4d9c33f-f46c-40e9-887b-b4a303669bdb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9"
Sep 07 04:06:12.254437 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.254347    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8dbf\" (UniqueName: \"kubernetes.io/projected/6103b3e4-bc89-4ccb-858b-0ba09f16aba1-kube-api-access-w8dbf\") pod \"payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t\" (UID: \"6103b3e4-bc89-4ccb-858b-0ba09f16aba1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t"
Sep 07 04:06:12.254595 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.254480    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6103b3e4-bc89-4ccb-858b-0ba09f16aba1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t\" (UID: \"6103b3e4-bc89-4ccb-858b-0ba09f16aba1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t"
Sep 07 04:06:12.255396 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.255374    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6103b3e4-bc89-4ccb-858b-0ba09f16aba1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t\" (UID: \"6103b3e4-bc89-4ccb-858b-0ba09f16aba1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t"
Sep 07 04:06:12.267415 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.267389    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8dbf\" (UniqueName: \"kubernetes.io/projected/6103b3e4-bc89-4ccb-858b-0ba09f16aba1-kube-api-access-w8dbf\") pod \"payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t\" (UID: \"6103b3e4-bc89-4ccb-858b-0ba09f16aba1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t"
Sep 07 04:06:12.350473 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.350435    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9"
Sep 07 04:06:12.396798 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.396757    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t"
Sep 07 04:06:12.627649 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.627563    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9"]
Sep 07 04:06:12.631309 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:06:12.631282    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4d9c33f_f46c_40e9_887b_b4a303669bdb.slice/crio-89a133c69e6bb0f9b683e365239aac0ee3e8a95cb4d7e96af234605893f00c88 WatchSource:0}: Error finding container 89a133c69e6bb0f9b683e365239aac0ee3e8a95cb4d7e96af234605893f00c88: Status 404 returned error can't find the container with id 89a133c69e6bb0f9b683e365239aac0ee3e8a95cb4d7e96af234605893f00c88
Sep 07 04:06:12.681428 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.681404    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t"]
Sep 07 04:06:12.683400 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:06:12.683376    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6103b3e4_bc89_4ccb_858b_0ba09f16aba1.slice/crio-d5b18b29982a4bb69d03a4d6e793fffe48c34ed5abf7b2bbfa477d60f48a7e4a WatchSource:0}: Error finding container d5b18b29982a4bb69d03a4d6e793fffe48c34ed5abf7b2bbfa477d60f48a7e4a: Status 404 returned error can't find the container with id d5b18b29982a4bb69d03a4d6e793fffe48c34ed5abf7b2bbfa477d60f48a7e4a
Sep 07 04:06:12.694006 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.693974    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t" event={"ID":"6103b3e4-bc89-4ccb-858b-0ba09f16aba1","Type":"ContainerStarted","Data":"d5b18b29982a4bb69d03a4d6e793fffe48c34ed5abf7b2bbfa477d60f48a7e4a"}
Sep 07 04:06:12.696127 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:12.696096    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9" event={"ID":"f4d9c33f-f46c-40e9-887b-b4a303669bdb","Type":"ContainerStarted","Data":"89a133c69e6bb0f9b683e365239aac0ee3e8a95cb4d7e96af234605893f00c88"}
Sep 07 04:06:13.703447 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:13.703413    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t" event={"ID":"6103b3e4-bc89-4ccb-858b-0ba09f16aba1","Type":"ContainerStarted","Data":"57ffd88f351506317126a4aec6d5399e5689dc7eeb80081f25690645a42f0833"}
Sep 07 04:06:13.703967 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:13.703480    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t"
Sep 07 04:06:13.704980 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:13.704955    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9" event={"ID":"f4d9c33f-f46c-40e9-887b-b4a303669bdb","Type":"ContainerStarted","Data":"45a6cffb74e4e86a19d70c0b95ba95c9a16a0b309b65835a0b44640a75a0d903"}
Sep 07 04:06:13.705051 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:13.705014    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9"
Sep 07 04:06:13.719862 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:13.719807    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t" podStartSLOduration=1.71978561 podStartE2EDuration="1.71978561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:06:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:06:13.718609372 +0000 UTC m=+1060.652090091" watchObservedRunningTime="2026-09-07 04:06:13.71978561 +0000 UTC m=+1060.653266319"
Sep 07 04:06:13.736389 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:13.736337    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9" podStartSLOduration=1.7363203029999998 podStartE2EDuration="1.736320303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:06:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:06:13.732705001 +0000 UTC m=+1060.666185722" watchObservedRunningTime="2026-09-07 04:06:13.736320303 +0000 UTC m=+1060.669801021"
Sep 07 04:06:22.083543 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:22.083507    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:23.874175 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:23.874133    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"]
Sep 07 04:06:23.878872 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:23.878849    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:23.883530 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:23.882657    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-b8cea6\"/\"openshift-service-ca.crt\""
Sep 07 04:06:23.883530 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:23.883012    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-b8cea6\"/\"default-dockercfg-vxkmj\""
Sep 07 04:06:23.883530 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:23.883286    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-b8cea6\"/\"auth-test-model-b8cea6-kserve-self-signed-certs\""
Sep 07 04:06:23.883810 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:23.883669    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-b8cea6\"/\"kube-root-ca.crt\""
Sep 07 04:06:23.887477 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:23.887447    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"]
Sep 07 04:06:23.906261 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:23.906225    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:23.968581 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:23.968542    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-home\") pod \"auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:23.968581 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:23.968587    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-dshm\") pod \"auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:23.968794 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:23.968619    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88f79824-2e2b-4748-82ba-60042f7e0157-tls-certs\") pod \"auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:23.968794 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:23.968685    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9j2h\" (UniqueName: \"kubernetes.io/projected/88f79824-2e2b-4748-82ba-60042f7e0157-kube-api-access-q9j2h\") pod \"auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:23.968794 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:23.968766    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-tmp-dir\") pod \"auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:23.968794 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:23.968789    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-model-cache\") pod \"auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:24.069889 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:24.069846    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-dshm\") pod \"auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:24.069889 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:24.069892    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88f79824-2e2b-4748-82ba-60042f7e0157-tls-certs\") pod \"auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:24.070166 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:24.069932    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9j2h\" (UniqueName: \"kubernetes.io/projected/88f79824-2e2b-4748-82ba-60042f7e0157-kube-api-access-q9j2h\") pod \"auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:24.070166 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:24.070032    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-tmp-dir\") pod \"auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:24.070166 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:24.070057    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-model-cache\") pod \"auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:24.070375 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:24.070192    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-home\") pod \"auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:24.070463 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:24.070441    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-model-cache\") pod \"auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:24.070519 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:24.070500    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-tmp-dir\") pod \"auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:24.070596 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:24.070570    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-home\") pod \"auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:24.072410 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:24.072392    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-dshm\") pod \"auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:24.072681 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:24.072666    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88f79824-2e2b-4748-82ba-60042f7e0157-tls-certs\") pod \"auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:24.077594 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:24.077576    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9j2h\" (UniqueName: \"kubernetes.io/projected/88f79824-2e2b-4748-82ba-60042f7e0157-kube-api-access-q9j2h\") pod \"auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:24.192993 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:24.192963    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:24.463857 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:24.463830    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"]
Sep 07 04:06:24.465474 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:06:24.465449    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88f79824_2e2b_4748_82ba_60042f7e0157.slice/crio-c8d805c14f0d507f8d55d7a2dc9c9ca0400ac69ea8aad98675ea6e79b4da96ce WatchSource:0}: Error finding container c8d805c14f0d507f8d55d7a2dc9c9ca0400ac69ea8aad98675ea6e79b4da96ce: Status 404 returned error can't find the container with id c8d805c14f0d507f8d55d7a2dc9c9ca0400ac69ea8aad98675ea6e79b4da96ce
Sep 07 04:06:24.711939 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:24.711863    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t"
Sep 07 04:06:24.712097 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:24.711940    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9"
Sep 07 04:06:24.766157 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:24.766061    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x" event={"ID":"88f79824-2e2b-4748-82ba-60042f7e0157","Type":"ContainerStarted","Data":"385e26b7c4f10e4dc1ba1cbc4b7752fbb6205b0f1a4f708805e8d31d9757896f"}
Sep 07 04:06:24.766157 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:24.766110    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x" event={"ID":"88f79824-2e2b-4748-82ba-60042f7e0157","Type":"ContainerStarted","Data":"c8d805c14f0d507f8d55d7a2dc9c9ca0400ac69ea8aad98675ea6e79b4da96ce"}
Sep 07 04:06:24.783195 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:24.783146    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x" podStartSLOduration=1.78313111 podStartE2EDuration="1.78313111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:06:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:06:24.78144701 +0000 UTC m=+1071.714927729" watchObservedRunningTime="2026-09-07 04:06:24.78313111 +0000 UTC m=+1071.716611863"
Sep 07 04:06:34.193331 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:34.193284    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:34.193771 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:34.193354    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:34.206539 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:34.206511    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:34.829273 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:34.829245    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:06:35.122126 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:35.122035    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:40.869686 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:40.869647    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"]
Sep 07 04:06:40.873494 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:40.873471    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:40.875993 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:40.875967    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-434d57\"/\"kube-root-ca.crt\""
Sep 07 04:06:40.877197 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:40.877173    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-434d57\"/\"default-dockercfg-7kj8j\""
Sep 07 04:06:40.877398 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:40.877235    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-434d57\"/\"openshift-service-ca.crt\""
Sep 07 04:06:40.877398 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:40.877294    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-434d57\"/\"auth-test-model-434d57-kserve-self-signed-certs\""
Sep 07 04:06:40.884718 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:40.884688    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"]
Sep 07 04:06:40.891002 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:40.890973    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:41.040348 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:41.040308    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-home\") pod \"auth-test-model-434d57-kserve-65cc5b79-q5jvk\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") " pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:41.040348 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:41.040348    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-model-cache\") pod \"auth-test-model-434d57-kserve-65cc5b79-q5jvk\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") " pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:41.040560 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:41.040394    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e7c1861-70cd-439a-bb77-941c9d7053f7-tls-certs\") pod \"auth-test-model-434d57-kserve-65cc5b79-q5jvk\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") " pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:41.040560 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:41.040475    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-tmp-dir\") pod \"auth-test-model-434d57-kserve-65cc5b79-q5jvk\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") " pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:41.040560 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:41.040496    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-dshm\") pod \"auth-test-model-434d57-kserve-65cc5b79-q5jvk\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") " pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:41.040560 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:41.040518    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj8wl\" (UniqueName: \"kubernetes.io/projected/5e7c1861-70cd-439a-bb77-941c9d7053f7-kube-api-access-bj8wl\") pod \"auth-test-model-434d57-kserve-65cc5b79-q5jvk\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") " pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:41.142030 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:41.141941    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-home\") pod \"auth-test-model-434d57-kserve-65cc5b79-q5jvk\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") " pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:41.142030 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:41.141977    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-model-cache\") pod \"auth-test-model-434d57-kserve-65cc5b79-q5jvk\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") " pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:41.142030 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:41.142010    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e7c1861-70cd-439a-bb77-941c9d7053f7-tls-certs\") pod \"auth-test-model-434d57-kserve-65cc5b79-q5jvk\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") " pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:41.142367 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:41.142038    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-tmp-dir\") pod \"auth-test-model-434d57-kserve-65cc5b79-q5jvk\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") " pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:41.142367 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:41.142065    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-dshm\") pod \"auth-test-model-434d57-kserve-65cc5b79-q5jvk\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") " pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:41.142367 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:41.142091    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj8wl\" (UniqueName: \"kubernetes.io/projected/5e7c1861-70cd-439a-bb77-941c9d7053f7-kube-api-access-bj8wl\") pod \"auth-test-model-434d57-kserve-65cc5b79-q5jvk\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") " pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:41.142525 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:41.142467    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-home\") pod \"auth-test-model-434d57-kserve-65cc5b79-q5jvk\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") " pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:41.142525 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:41.142501    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-model-cache\") pod \"auth-test-model-434d57-kserve-65cc5b79-q5jvk\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") " pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:41.142596 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:41.142539    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-tmp-dir\") pod \"auth-test-model-434d57-kserve-65cc5b79-q5jvk\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") " pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:41.144623 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:41.144596    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-dshm\") pod \"auth-test-model-434d57-kserve-65cc5b79-q5jvk\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") " pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:41.144889 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:41.144870    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e7c1861-70cd-439a-bb77-941c9d7053f7-tls-certs\") pod \"auth-test-model-434d57-kserve-65cc5b79-q5jvk\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") " pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:41.150444 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:41.150419    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj8wl\" (UniqueName: \"kubernetes.io/projected/5e7c1861-70cd-439a-bb77-941c9d7053f7-kube-api-access-bj8wl\") pod \"auth-test-model-434d57-kserve-65cc5b79-q5jvk\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") " pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:41.186150 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:41.186116    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:41.460774 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:41.460748    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"]
Sep 07 04:06:41.462732 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:06:41.462700    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e7c1861_70cd_439a_bb77_941c9d7053f7.slice/crio-e32b60f6751237ce68cae1671acf2dfd552b31494b3f31bd6cd3f90f56eb0e46 WatchSource:0}: Error finding container e32b60f6751237ce68cae1671acf2dfd552b31494b3f31bd6cd3f90f56eb0e46: Status 404 returned error can't find the container with id e32b60f6751237ce68cae1671acf2dfd552b31494b3f31bd6cd3f90f56eb0e46
Sep 07 04:06:41.858277 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:41.858155    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk" event={"ID":"5e7c1861-70cd-439a-bb77-941c9d7053f7","Type":"ContainerStarted","Data":"cc7ec851713998098f9555d588d7b410d28d9cf2dd9281e532a382b95de30ec5"}
Sep 07 04:06:41.858277 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:41.858199    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk" event={"ID":"5e7c1861-70cd-439a-bb77-941c9d7053f7","Type":"ContainerStarted","Data":"e32b60f6751237ce68cae1671acf2dfd552b31494b3f31bd6cd3f90f56eb0e46"}
Sep 07 04:06:41.876938 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:41.876873    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk" podStartSLOduration=1.876857785 podStartE2EDuration="1.876857785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:06:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:06:41.875564231 +0000 UTC m=+1088.809044960" watchObservedRunningTime="2026-09-07 04:06:41.876857785 +0000 UTC m=+1088.810338503"
Sep 07 04:06:48.521348 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:48.521294    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:48.811825 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:48.811724    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:49.419557 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:49.419505    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:51.187264 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:51.187226    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:51.187722 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:51.187490    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:51.200491 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:51.200467    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:51.652144 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:51.652043    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:51.745916 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:51.745884    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:51.921490 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:51.921401    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:06:52.929334 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:52.929300    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:52.991595 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:52.991550    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:53.051705 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:53.051672    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:53.369098 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:53.369056    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:06:53.449511 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:06:53.449467    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:07:16.647175 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:16.647126    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:07:17.401276 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:17.401240    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:07:20.072909 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.072878    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"]
Sep 07 04:07:20.079020 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.079000    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:20.081540 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.081507    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-107fc16f\"/\"openshift-service-ca.crt\""
Sep 07 04:07:20.082635 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.082588    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-107fc16f\"/\"kube-root-ca.crt\""
Sep 07 04:07:20.082781 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.082659    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-107fc16f\"/\"e2e-lifecycle-model-107fc16f-kserve-self-signed-certs\""
Sep 07 04:07:20.082781 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.082601    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-107fc16f\"/\"default-dockercfg-pzbx4\""
Sep 07 04:07:20.086203 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.086178    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"]
Sep 07 04:07:20.090069 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.090040    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:07:20.230909 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.230874    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-dshm\") pod \"e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") " pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:20.231090 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.230942    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-model-cache\") pod \"e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") " pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:20.231090 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.230987    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4212509f-6cb6-44ad-8e86-c805e504ba2d-tls-certs\") pod \"e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") " pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:20.231090 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.231035    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-tmp-dir\") pod \"e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") " pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:20.231090 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.231069    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlxdn\" (UniqueName: \"kubernetes.io/projected/4212509f-6cb6-44ad-8e86-c805e504ba2d-kube-api-access-mlxdn\") pod \"e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") " pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:20.231276 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.231144    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-home\") pod \"e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") " pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:20.332268 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.332160    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-dshm\") pod \"e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") " pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:20.332268 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.332229    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-model-cache\") pod \"e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") " pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:20.332268 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.332253    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4212509f-6cb6-44ad-8e86-c805e504ba2d-tls-certs\") pod \"e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") " pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:20.332569 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.332283    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-tmp-dir\") pod \"e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") " pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:20.332569 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.332310    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlxdn\" (UniqueName: \"kubernetes.io/projected/4212509f-6cb6-44ad-8e86-c805e504ba2d-kube-api-access-mlxdn\") pod \"e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") " pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:20.332569 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.332337    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-home\") pod \"e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") " pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:20.332732 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.332690    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-model-cache\") pod \"e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") " pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:20.332777 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.332725    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-tmp-dir\") pod \"e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") " pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:20.332777 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.332750    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-home\") pod \"e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") " pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:20.334641 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.334618    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-dshm\") pod \"e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") " pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:20.334852 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.334833    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4212509f-6cb6-44ad-8e86-c805e504ba2d-tls-certs\") pod \"e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") " pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:20.339915 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.339891    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlxdn\" (UniqueName: \"kubernetes.io/projected/4212509f-6cb6-44ad-8e86-c805e504ba2d-kube-api-access-mlxdn\") pod \"e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") " pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:20.390264 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.390200    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:20.665009 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:20.664981    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"]
Sep 07 04:07:20.666881 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:07:20.666849    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4212509f_6cb6_44ad_8e86_c805e504ba2d.slice/crio-258340f2d88856771b4bc2ff39e9ce3a96b7207fb8a0ca0f3a38c9b72cb888e4 WatchSource:0}: Error finding container 258340f2d88856771b4bc2ff39e9ce3a96b7207fb8a0ca0f3a38c9b72cb888e4: Status 404 returned error can't find the container with id 258340f2d88856771b4bc2ff39e9ce3a96b7207fb8a0ca0f3a38c9b72cb888e4
Sep 07 04:07:21.079860 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:21.079825    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc" event={"ID":"4212509f-6cb6-44ad-8e86-c805e504ba2d","Type":"ContainerStarted","Data":"12ad87610d34196b7bcad920c5095095f2f509b8c4f0b65e05b027f9caed8e8e"}
Sep 07 04:07:21.080251 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:21.079870    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc" event={"ID":"4212509f-6cb6-44ad-8e86-c805e504ba2d","Type":"ContainerStarted","Data":"258340f2d88856771b4bc2ff39e9ce3a96b7207fb8a0ca0f3a38c9b72cb888e4"}
Sep 07 04:07:21.096170 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:21.096116    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc" podStartSLOduration=1.096100458 podStartE2EDuration="1.096100458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:07:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:07:21.093696919 +0000 UTC m=+1128.027177639" watchObservedRunningTime="2026-09-07 04:07:21.096100458 +0000 UTC m=+1128.029581175"
Sep 07 04:07:30.391056 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:30.391016    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:30.391056 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:30.391068    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:30.404305 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:30.404269    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:31.145320 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:31.145287    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:07:50.974274 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:07:50.974193    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:08:06.374570 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:06.374439    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:08:06.442991 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:06.442960    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:08:06.505928 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:06.505900    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:08:06.606620 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:06.606577    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:08:09.820920 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:09.820883    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:08:14.800759 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:14.800721    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"]
Sep 07 04:08:14.840446 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:14.840410    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:08:16.262002 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:16.261946    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:08:16.317767 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:16.317731    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:08:29.821950 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:29.821906    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc" podUID="4212509f-6cb6-44ad-8e86-c805e504ba2d" containerName="main" containerID="cri-o://12ad87610d34196b7bcad920c5095095f2f509b8c4f0b65e05b027f9caed8e8e" gracePeriod=45
Sep 07 04:08:30.071644 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.071620    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:08:30.218864 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.218827    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-dshm\") pod \"4212509f-6cb6-44ad-8e86-c805e504ba2d\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") "
Sep 07 04:08:30.219076 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.218926    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-model-cache\") pod \"4212509f-6cb6-44ad-8e86-c805e504ba2d\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") "
Sep 07 04:08:30.219076 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.218953    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4212509f-6cb6-44ad-8e86-c805e504ba2d-tls-certs\") pod \"4212509f-6cb6-44ad-8e86-c805e504ba2d\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") "
Sep 07 04:08:30.219076 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.218974    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-tmp-dir\") pod \"4212509f-6cb6-44ad-8e86-c805e504ba2d\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") "
Sep 07 04:08:30.219076 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.219001    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlxdn\" (UniqueName: \"kubernetes.io/projected/4212509f-6cb6-44ad-8e86-c805e504ba2d-kube-api-access-mlxdn\") pod \"4212509f-6cb6-44ad-8e86-c805e504ba2d\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") "
Sep 07 04:08:30.219354 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.219139    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-home\") pod \"4212509f-6cb6-44ad-8e86-c805e504ba2d\" (UID: \"4212509f-6cb6-44ad-8e86-c805e504ba2d\") "
Sep 07 04:08:30.219354 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.219278    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-model-cache" (OuterVolumeSpecName: "model-cache") pod "4212509f-6cb6-44ad-8e86-c805e504ba2d" (UID: "4212509f-6cb6-44ad-8e86-c805e504ba2d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:08:30.219354 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.219293    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4212509f-6cb6-44ad-8e86-c805e504ba2d" (UID: "4212509f-6cb6-44ad-8e86-c805e504ba2d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:08:30.219505 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.219393    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-home" (OuterVolumeSpecName: "home") pod "4212509f-6cb6-44ad-8e86-c805e504ba2d" (UID: "4212509f-6cb6-44ad-8e86-c805e504ba2d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:08:30.219777 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.219590    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-tmp-dir\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:08:30.219777 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.219616    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-home\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:08:30.219777 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.219670    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-model-cache\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:08:30.221374 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.221349    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4212509f-6cb6-44ad-8e86-c805e504ba2d-kube-api-access-mlxdn" (OuterVolumeSpecName: "kube-api-access-mlxdn") pod "4212509f-6cb6-44ad-8e86-c805e504ba2d" (UID: "4212509f-6cb6-44ad-8e86-c805e504ba2d"). InnerVolumeSpecName "kube-api-access-mlxdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:08:30.221374 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.221361    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4212509f-6cb6-44ad-8e86-c805e504ba2d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4212509f-6cb6-44ad-8e86-c805e504ba2d" (UID: "4212509f-6cb6-44ad-8e86-c805e504ba2d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:08:30.221512 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.221428    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-dshm" (OuterVolumeSpecName: "dshm") pod "4212509f-6cb6-44ad-8e86-c805e504ba2d" (UID: "4212509f-6cb6-44ad-8e86-c805e504ba2d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:08:30.320409 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.320372    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlxdn\" (UniqueName: \"kubernetes.io/projected/4212509f-6cb6-44ad-8e86-c805e504ba2d-kube-api-access-mlxdn\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:08:30.320409 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.320402    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4212509f-6cb6-44ad-8e86-c805e504ba2d-dshm\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:08:30.320409 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.320415    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4212509f-6cb6-44ad-8e86-c805e504ba2d-tls-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:08:30.436815 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.436786    2583 generic.go:356] "Generic (PLEG): container finished" podID="4212509f-6cb6-44ad-8e86-c805e504ba2d" containerID="12ad87610d34196b7bcad920c5095095f2f509b8c4f0b65e05b027f9caed8e8e" exitCode=0
Sep 07 04:08:30.437011 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.436832    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"
Sep 07 04:08:30.437011 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.436874    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc" event={"ID":"4212509f-6cb6-44ad-8e86-c805e504ba2d","Type":"ContainerDied","Data":"12ad87610d34196b7bcad920c5095095f2f509b8c4f0b65e05b027f9caed8e8e"}
Sep 07 04:08:30.437011 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.436917    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc" event={"ID":"4212509f-6cb6-44ad-8e86-c805e504ba2d","Type":"ContainerDied","Data":"258340f2d88856771b4bc2ff39e9ce3a96b7207fb8a0ca0f3a38c9b72cb888e4"}
Sep 07 04:08:30.437011 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.436937    2583 scope.go:117] "RemoveContainer" containerID="12ad87610d34196b7bcad920c5095095f2f509b8c4f0b65e05b027f9caed8e8e"
Sep 07 04:08:30.447352 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.447333    2583 scope.go:117] "RemoveContainer" containerID="12ad87610d34196b7bcad920c5095095f2f509b8c4f0b65e05b027f9caed8e8e"
Sep 07 04:08:30.447667 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:08:30.447645    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12ad87610d34196b7bcad920c5095095f2f509b8c4f0b65e05b027f9caed8e8e\": container with ID starting with 12ad87610d34196b7bcad920c5095095f2f509b8c4f0b65e05b027f9caed8e8e not found: ID does not exist" containerID="12ad87610d34196b7bcad920c5095095f2f509b8c4f0b65e05b027f9caed8e8e"
Sep 07 04:08:30.447775 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.447674    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12ad87610d34196b7bcad920c5095095f2f509b8c4f0b65e05b027f9caed8e8e"} err="failed to get container status \"12ad87610d34196b7bcad920c5095095f2f509b8c4f0b65e05b027f9caed8e8e\": rpc error: code = NotFound desc = could not find container \"12ad87610d34196b7bcad920c5095095f2f509b8c4f0b65e05b027f9caed8e8e\": container with ID starting with 12ad87610d34196b7bcad920c5095095f2f509b8c4f0b65e05b027f9caed8e8e not found: ID does not exist"
Sep 07 04:08:30.458658 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.458623    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"]
Sep 07 04:08:30.470054 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:30.469983    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-107fc16f/e2e-lifecycle-model-107fc16f-kserve-69d4ffc747-vl6cc"]
Sep 07 04:08:31.521955 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:31.521922    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4212509f-6cb6-44ad-8e86-c805e504ba2d" path="/var/lib/kubelet/pods/4212509f-6cb6-44ad-8e86-c805e504ba2d/volumes"
Sep 07 04:08:51.881879 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:51.881843    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:08:53.117904 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:53.117874    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:08:55.100418 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:55.100379    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"]
Sep 07 04:08:55.109272 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:55.109237    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:08:56.062172 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:56.062141    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"]
Sep 07 04:08:56.071314 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:56.071283    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:08:57.540336 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:57.540303    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:08:57.606455 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:57.606422    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:08:57.789204 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:57.789160    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:08:59.012954 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:59.012914    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:08:59.112938 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:59.112905    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:08:59.212626 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:59.212591    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:08:59.817283 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:59.817255    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:08:59.912115 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:08:59.912072    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:00.011434 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:00.011407    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:01.256486 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:01.256438    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:04.703278 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:04.703205    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:06.986858 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:06.986819    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:07.054987 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:07.054945    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:07.669352 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:07.669321    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:10.122083 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.122042    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x" podUID="88f79824-2e2b-4748-82ba-60042f7e0157" containerName="main" containerID="cri-o://385e26b7c4f10e4dc1ba1cbc4b7752fbb6205b0f1a4f708805e8d31d9757896f" gracePeriod=45
Sep 07 04:09:10.371188 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.371154    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:09:10.518330 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.518292    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-dshm\") pod \"88f79824-2e2b-4748-82ba-60042f7e0157\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") "
Sep 07 04:09:10.518511 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.518379    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-tmp-dir\") pod \"88f79824-2e2b-4748-82ba-60042f7e0157\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") "
Sep 07 04:09:10.518511 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.518407    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9j2h\" (UniqueName: \"kubernetes.io/projected/88f79824-2e2b-4748-82ba-60042f7e0157-kube-api-access-q9j2h\") pod \"88f79824-2e2b-4748-82ba-60042f7e0157\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") "
Sep 07 04:09:10.518511 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.518442    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-model-cache\") pod \"88f79824-2e2b-4748-82ba-60042f7e0157\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") "
Sep 07 04:09:10.518511 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.518465    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-home\") pod \"88f79824-2e2b-4748-82ba-60042f7e0157\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") "
Sep 07 04:09:10.518511 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.518490    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88f79824-2e2b-4748-82ba-60042f7e0157-tls-certs\") pod \"88f79824-2e2b-4748-82ba-60042f7e0157\" (UID: \"88f79824-2e2b-4748-82ba-60042f7e0157\") "
Sep 07 04:09:10.518774 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.518650    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "88f79824-2e2b-4748-82ba-60042f7e0157" (UID: "88f79824-2e2b-4748-82ba-60042f7e0157"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:09:10.518774 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.518745    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-model-cache" (OuterVolumeSpecName: "model-cache") pod "88f79824-2e2b-4748-82ba-60042f7e0157" (UID: "88f79824-2e2b-4748-82ba-60042f7e0157"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:09:10.518862 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.518768    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-home" (OuterVolumeSpecName: "home") pod "88f79824-2e2b-4748-82ba-60042f7e0157" (UID: "88f79824-2e2b-4748-82ba-60042f7e0157"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:09:10.518862 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.518803    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-tmp-dir\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:09:10.520618 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.520595    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-dshm" (OuterVolumeSpecName: "dshm") pod "88f79824-2e2b-4748-82ba-60042f7e0157" (UID: "88f79824-2e2b-4748-82ba-60042f7e0157"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:09:10.520793 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.520766    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88f79824-2e2b-4748-82ba-60042f7e0157-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "88f79824-2e2b-4748-82ba-60042f7e0157" (UID: "88f79824-2e2b-4748-82ba-60042f7e0157"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:09:10.520887 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.520870    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88f79824-2e2b-4748-82ba-60042f7e0157-kube-api-access-q9j2h" (OuterVolumeSpecName: "kube-api-access-q9j2h") pod "88f79824-2e2b-4748-82ba-60042f7e0157" (UID: "88f79824-2e2b-4748-82ba-60042f7e0157"). InnerVolumeSpecName "kube-api-access-q9j2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:09:10.620267 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.620173    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-model-cache\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:09:10.620267 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.620263    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-home\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:09:10.620267 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.620275    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88f79824-2e2b-4748-82ba-60042f7e0157-tls-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:09:10.620573 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.620288    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88f79824-2e2b-4748-82ba-60042f7e0157-dshm\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:09:10.620573 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.620300    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9j2h\" (UniqueName: \"kubernetes.io/projected/88f79824-2e2b-4748-82ba-60042f7e0157-kube-api-access-q9j2h\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:09:10.642717 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.642683    2583 generic.go:356] "Generic (PLEG): container finished" podID="88f79824-2e2b-4748-82ba-60042f7e0157" containerID="385e26b7c4f10e4dc1ba1cbc4b7752fbb6205b0f1a4f708805e8d31d9757896f" exitCode=0
Sep 07 04:09:10.642886 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.642729    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"
Sep 07 04:09:10.642886 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.642769    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x" event={"ID":"88f79824-2e2b-4748-82ba-60042f7e0157","Type":"ContainerDied","Data":"385e26b7c4f10e4dc1ba1cbc4b7752fbb6205b0f1a4f708805e8d31d9757896f"}
Sep 07 04:09:10.642886 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.642810    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x" event={"ID":"88f79824-2e2b-4748-82ba-60042f7e0157","Type":"ContainerDied","Data":"c8d805c14f0d507f8d55d7a2dc9c9ca0400ac69ea8aad98675ea6e79b4da96ce"}
Sep 07 04:09:10.642886 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.642827    2583 scope.go:117] "RemoveContainer" containerID="385e26b7c4f10e4dc1ba1cbc4b7752fbb6205b0f1a4f708805e8d31d9757896f"
Sep 07 04:09:10.652921 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.652904    2583 scope.go:117] "RemoveContainer" containerID="385e26b7c4f10e4dc1ba1cbc4b7752fbb6205b0f1a4f708805e8d31d9757896f"
Sep 07 04:09:10.653237 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:09:10.653192    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"385e26b7c4f10e4dc1ba1cbc4b7752fbb6205b0f1a4f708805e8d31d9757896f\": container with ID starting with 385e26b7c4f10e4dc1ba1cbc4b7752fbb6205b0f1a4f708805e8d31d9757896f not found: ID does not exist" containerID="385e26b7c4f10e4dc1ba1cbc4b7752fbb6205b0f1a4f708805e8d31d9757896f"
Sep 07 04:09:10.653302 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.653242    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"385e26b7c4f10e4dc1ba1cbc4b7752fbb6205b0f1a4f708805e8d31d9757896f"} err="failed to get container status \"385e26b7c4f10e4dc1ba1cbc4b7752fbb6205b0f1a4f708805e8d31d9757896f\": rpc error: code = NotFound desc = could not find container \"385e26b7c4f10e4dc1ba1cbc4b7752fbb6205b0f1a4f708805e8d31d9757896f\": container with ID starting with 385e26b7c4f10e4dc1ba1cbc4b7752fbb6205b0f1a4f708805e8d31d9757896f not found: ID does not exist"
Sep 07 04:09:10.663424 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.663403    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"]
Sep 07 04:09:10.667152 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:10.667130    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b8cea6/auth-test-model-b8cea6-kserve-7c7b8d94dc-xsc6x"]
Sep 07 04:09:11.084451 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.084412    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk" podUID="5e7c1861-70cd-439a-bb77-941c9d7053f7" containerName="main" containerID="cri-o://cc7ec851713998098f9555d588d7b410d28d9cf2dd9281e532a382b95de30ec5" gracePeriod=45
Sep 07 04:09:11.340185 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.340127    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:09:11.427288 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.427249    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-dshm\") pod \"5e7c1861-70cd-439a-bb77-941c9d7053f7\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") "
Sep 07 04:09:11.427486 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.427307    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-home\") pod \"5e7c1861-70cd-439a-bb77-941c9d7053f7\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") "
Sep 07 04:09:11.427486 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.427354    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-model-cache\") pod \"5e7c1861-70cd-439a-bb77-941c9d7053f7\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") "
Sep 07 04:09:11.427486 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.427388    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e7c1861-70cd-439a-bb77-941c9d7053f7-tls-certs\") pod \"5e7c1861-70cd-439a-bb77-941c9d7053f7\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") "
Sep 07 04:09:11.427486 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.427456    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bj8wl\" (UniqueName: \"kubernetes.io/projected/5e7c1861-70cd-439a-bb77-941c9d7053f7-kube-api-access-bj8wl\") pod \"5e7c1861-70cd-439a-bb77-941c9d7053f7\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") "
Sep 07 04:09:11.427486 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.427484    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-tmp-dir\") pod \"5e7c1861-70cd-439a-bb77-941c9d7053f7\" (UID: \"5e7c1861-70cd-439a-bb77-941c9d7053f7\") "
Sep 07 04:09:11.427750 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.427563    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-home" (OuterVolumeSpecName: "home") pod "5e7c1861-70cd-439a-bb77-941c9d7053f7" (UID: "5e7c1861-70cd-439a-bb77-941c9d7053f7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:09:11.427750 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.427602    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-model-cache" (OuterVolumeSpecName: "model-cache") pod "5e7c1861-70cd-439a-bb77-941c9d7053f7" (UID: "5e7c1861-70cd-439a-bb77-941c9d7053f7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:09:11.427866 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.427789    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-home\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:09:11.427866 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.427803    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-model-cache\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:09:11.427866 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.427855    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5e7c1861-70cd-439a-bb77-941c9d7053f7" (UID: "5e7c1861-70cd-439a-bb77-941c9d7053f7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:09:11.429768 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.429742    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-dshm" (OuterVolumeSpecName: "dshm") pod "5e7c1861-70cd-439a-bb77-941c9d7053f7" (UID: "5e7c1861-70cd-439a-bb77-941c9d7053f7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:09:11.429881 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.429803    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e7c1861-70cd-439a-bb77-941c9d7053f7-kube-api-access-bj8wl" (OuterVolumeSpecName: "kube-api-access-bj8wl") pod "5e7c1861-70cd-439a-bb77-941c9d7053f7" (UID: "5e7c1861-70cd-439a-bb77-941c9d7053f7"). InnerVolumeSpecName "kube-api-access-bj8wl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:09:11.429941 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.429889    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e7c1861-70cd-439a-bb77-941c9d7053f7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5e7c1861-70cd-439a-bb77-941c9d7053f7" (UID: "5e7c1861-70cd-439a-bb77-941c9d7053f7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:09:11.522153 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.522117    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88f79824-2e2b-4748-82ba-60042f7e0157" path="/var/lib/kubelet/pods/88f79824-2e2b-4748-82ba-60042f7e0157/volumes"
Sep 07 04:09:11.528327 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.528307    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-dshm\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:09:11.528327 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.528326    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e7c1861-70cd-439a-bb77-941c9d7053f7-tls-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:09:11.528495 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.528336    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bj8wl\" (UniqueName: \"kubernetes.io/projected/5e7c1861-70cd-439a-bb77-941c9d7053f7-kube-api-access-bj8wl\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:09:11.528495 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.528345    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e7c1861-70cd-439a-bb77-941c9d7053f7-tmp-dir\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:09:11.650843 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.650752    2583 generic.go:356] "Generic (PLEG): container finished" podID="5e7c1861-70cd-439a-bb77-941c9d7053f7" containerID="cc7ec851713998098f9555d588d7b410d28d9cf2dd9281e532a382b95de30ec5" exitCode=0
Sep 07 04:09:11.650843 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.650800    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"
Sep 07 04:09:11.650843 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.650835    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk" event={"ID":"5e7c1861-70cd-439a-bb77-941c9d7053f7","Type":"ContainerDied","Data":"cc7ec851713998098f9555d588d7b410d28d9cf2dd9281e532a382b95de30ec5"}
Sep 07 04:09:11.651099 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.650879    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk" event={"ID":"5e7c1861-70cd-439a-bb77-941c9d7053f7","Type":"ContainerDied","Data":"e32b60f6751237ce68cae1671acf2dfd552b31494b3f31bd6cd3f90f56eb0e46"}
Sep 07 04:09:11.651099 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.650909    2583 scope.go:117] "RemoveContainer" containerID="cc7ec851713998098f9555d588d7b410d28d9cf2dd9281e532a382b95de30ec5"
Sep 07 04:09:11.659785 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.659765    2583 scope.go:117] "RemoveContainer" containerID="cc7ec851713998098f9555d588d7b410d28d9cf2dd9281e532a382b95de30ec5"
Sep 07 04:09:11.660158 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:09:11.660134    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc7ec851713998098f9555d588d7b410d28d9cf2dd9281e532a382b95de30ec5\": container with ID starting with cc7ec851713998098f9555d588d7b410d28d9cf2dd9281e532a382b95de30ec5 not found: ID does not exist" containerID="cc7ec851713998098f9555d588d7b410d28d9cf2dd9281e532a382b95de30ec5"
Sep 07 04:09:11.660293 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.660163    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc7ec851713998098f9555d588d7b410d28d9cf2dd9281e532a382b95de30ec5"} err="failed to get container status \"cc7ec851713998098f9555d588d7b410d28d9cf2dd9281e532a382b95de30ec5\": rpc error: code = NotFound desc = could not find container \"cc7ec851713998098f9555d588d7b410d28d9cf2dd9281e532a382b95de30ec5\": container with ID starting with cc7ec851713998098f9555d588d7b410d28d9cf2dd9281e532a382b95de30ec5 not found: ID does not exist"
Sep 07 04:09:11.690169 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.690133    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"]
Sep 07 04:09:11.703938 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:11.703910    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-434d57/auth-test-model-434d57-kserve-65cc5b79-q5jvk"]
Sep 07 04:09:13.374609 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:13.374568    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:13.477173 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:13.477138    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:13.522651 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:13.522615    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e7c1861-70cd-439a-bb77-941c9d7053f7" path="/var/lib/kubelet/pods/5e7c1861-70cd-439a-bb77-941c9d7053f7/volumes"
Sep 07 04:09:13.574192 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:13.574157    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:24.080127 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:24.080091    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:24.305506 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:24.305471    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:27.083834 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.083753    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"]
Sep 07 04:09:27.084430 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.084411    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4212509f-6cb6-44ad-8e86-c805e504ba2d" containerName="main"
Sep 07 04:09:27.084518 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.084434    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4212509f-6cb6-44ad-8e86-c805e504ba2d" containerName="main"
Sep 07 04:09:27.084518 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.084456    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e7c1861-70cd-439a-bb77-941c9d7053f7" containerName="main"
Sep 07 04:09:27.084518 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.084464    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e7c1861-70cd-439a-bb77-941c9d7053f7" containerName="main"
Sep 07 04:09:27.084518 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.084481    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88f79824-2e2b-4748-82ba-60042f7e0157" containerName="main"
Sep 07 04:09:27.084518 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.084488    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="88f79824-2e2b-4748-82ba-60042f7e0157" containerName="main"
Sep 07 04:09:27.084764 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.084612    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="88f79824-2e2b-4748-82ba-60042f7e0157" containerName="main"
Sep 07 04:09:27.084764 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.084625    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e7c1861-70cd-439a-bb77-941c9d7053f7" containerName="main"
Sep 07 04:09:27.084764 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.084636    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4212509f-6cb6-44ad-8e86-c805e504ba2d" containerName="main"
Sep 07 04:09:27.089378 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.089356    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:27.092137 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.092076    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-434d57\"/\"default-dockercfg-7kj8j\""
Sep 07 04:09:27.092137 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.092133    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-434d57\"/\"openshift-service-ca.crt\""
Sep 07 04:09:27.094073 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.093680    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-434d57\"/\"kube-root-ca.crt\""
Sep 07 04:09:27.094073 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.093941    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-434d57\"/\"sub-test-model-434d57-kserve-self-signed-certs\""
Sep 07 04:09:27.097618 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.097594    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"]
Sep 07 04:09:27.104795 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.104730    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:27.169983 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.169953    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-tmp-dir\") pod \"sub-test-model-434d57-kserve-7fd6484df-dxhhq\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") " pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:27.169983 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.169988    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-dshm\") pod \"sub-test-model-434d57-kserve-7fd6484df-dxhhq\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") " pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:27.170199 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.170033    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvbzr\" (UniqueName: \"kubernetes.io/projected/d718aa52-da42-417f-b62a-a3ca55cf99cb-kube-api-access-gvbzr\") pod \"sub-test-model-434d57-kserve-7fd6484df-dxhhq\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") " pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:27.170199 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.170102    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-home\") pod \"sub-test-model-434d57-kserve-7fd6484df-dxhhq\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") " pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:27.170199 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.170132    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-model-cache\") pod \"sub-test-model-434d57-kserve-7fd6484df-dxhhq\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") " pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:27.170199 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.170161    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d718aa52-da42-417f-b62a-a3ca55cf99cb-tls-certs\") pod \"sub-test-model-434d57-kserve-7fd6484df-dxhhq\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") " pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:27.271486 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.271455    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvbzr\" (UniqueName: \"kubernetes.io/projected/d718aa52-da42-417f-b62a-a3ca55cf99cb-kube-api-access-gvbzr\") pod \"sub-test-model-434d57-kserve-7fd6484df-dxhhq\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") " pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:27.271658 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.271508    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-home\") pod \"sub-test-model-434d57-kserve-7fd6484df-dxhhq\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") " pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:27.271658 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.271541    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-model-cache\") pod \"sub-test-model-434d57-kserve-7fd6484df-dxhhq\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") " pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:27.271658 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.271580    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d718aa52-da42-417f-b62a-a3ca55cf99cb-tls-certs\") pod \"sub-test-model-434d57-kserve-7fd6484df-dxhhq\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") " pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:27.271658 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.271614    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-tmp-dir\") pod \"sub-test-model-434d57-kserve-7fd6484df-dxhhq\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") " pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:27.271658 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.271649    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-dshm\") pod \"sub-test-model-434d57-kserve-7fd6484df-dxhhq\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") " pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:27.272001 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.271975    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-model-cache\") pod \"sub-test-model-434d57-kserve-7fd6484df-dxhhq\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") " pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:27.272001 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.271990    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-home\") pod \"sub-test-model-434d57-kserve-7fd6484df-dxhhq\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") " pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:27.272121 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.272056    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-tmp-dir\") pod \"sub-test-model-434d57-kserve-7fd6484df-dxhhq\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") " pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:27.273982 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.273964    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-dshm\") pod \"sub-test-model-434d57-kserve-7fd6484df-dxhhq\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") " pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:27.274264 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.274249    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d718aa52-da42-417f-b62a-a3ca55cf99cb-tls-certs\") pod \"sub-test-model-434d57-kserve-7fd6484df-dxhhq\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") " pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:27.279309 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.279275    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvbzr\" (UniqueName: \"kubernetes.io/projected/d718aa52-da42-417f-b62a-a3ca55cf99cb-kube-api-access-gvbzr\") pod \"sub-test-model-434d57-kserve-7fd6484df-dxhhq\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") " pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:27.405276 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.405166    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:27.678815 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.678789    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"]
Sep 07 04:09:27.680589 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:09:27.680564    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd718aa52_da42_417f_b62a_a3ca55cf99cb.slice/crio-3e1de1e3f17a5eeff9246641090de9f911e8577010d1c375dad13069d8dafb4a WatchSource:0}: Error finding container 3e1de1e3f17a5eeff9246641090de9f911e8577010d1c375dad13069d8dafb4a: Status 404 returned error can't find the container with id 3e1de1e3f17a5eeff9246641090de9f911e8577010d1c375dad13069d8dafb4a
Sep 07 04:09:27.735742 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:27.735709    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq" event={"ID":"d718aa52-da42-417f-b62a-a3ca55cf99cb","Type":"ContainerStarted","Data":"3e1de1e3f17a5eeff9246641090de9f911e8577010d1c375dad13069d8dafb4a"}
Sep 07 04:09:28.742580 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:28.742547    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq" event={"ID":"d718aa52-da42-417f-b62a-a3ca55cf99cb","Type":"ContainerStarted","Data":"eb8e95604450fa152da21bce985f3d5101784258e61ae01a89416b1976d9c549"}
Sep 07 04:09:28.758166 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:28.758107    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq" podStartSLOduration=1.75808609 podStartE2EDuration="1.75808609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:09:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:09:28.757823264 +0000 UTC m=+1255.691303983" watchObservedRunningTime="2026-09-07 04:09:28.75808609 +0000 UTC m=+1255.691566807"
Sep 07 04:09:37.406395 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:37.406357    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:37.406812 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:37.406410    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:37.419258 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:37.419207    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:37.806234 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:37.806184    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:09:45.227329 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:45.227296    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:51.017426 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:51.017380    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:51.117489 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:51.117439    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:51.358467 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:51.358380    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:51.678933 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:51.678785    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:51.781699 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:51.781671    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:54.737818 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:54.737766    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:54.832584 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:54.832532    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:56.007254 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:56.007226    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:56.066938 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:56.066905    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:56.345244 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:56.345081    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:56.449628 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:56.449596    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:56.543571 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:56.543540    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:09:56.548166 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:56.548140    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb"]
Sep 07 04:09:56.553190 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:56.553175    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb"
Sep 07 04:09:56.556073 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:56.556055    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-467e44-dockercfg-g64tg\""
Sep 07 04:09:56.556372 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:56.556357    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-467e44\""
Sep 07 04:09:56.566814 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:56.566789    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb"]
Sep 07 04:09:56.647093 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:56.647007    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqkfq\" (UniqueName: \"kubernetes.io/projected/daa7327d-1d16-4d4a-b8ca-bf1e86d0af98-kube-api-access-fqkfq\") pod \"payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb\" (UID: \"daa7327d-1d16-4d4a-b8ca-bf1e86d0af98\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb"
Sep 07 04:09:56.647093 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:56.647077    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daa7327d-1d16-4d4a-b8ca-bf1e86d0af98-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb\" (UID: \"daa7327d-1d16-4d4a-b8ca-bf1e86d0af98\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb"
Sep 07 04:09:56.747686 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:56.747647    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqkfq\" (UniqueName: \"kubernetes.io/projected/daa7327d-1d16-4d4a-b8ca-bf1e86d0af98-kube-api-access-fqkfq\") pod \"payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb\" (UID: \"daa7327d-1d16-4d4a-b8ca-bf1e86d0af98\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb"
Sep 07 04:09:56.747874 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:56.747746    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daa7327d-1d16-4d4a-b8ca-bf1e86d0af98-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb\" (UID: \"daa7327d-1d16-4d4a-b8ca-bf1e86d0af98\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb"
Sep 07 04:09:56.748461 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:56.748443    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daa7327d-1d16-4d4a-b8ca-bf1e86d0af98-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb\" (UID: \"daa7327d-1d16-4d4a-b8ca-bf1e86d0af98\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb"
Sep 07 04:09:56.764160 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:56.764125    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqkfq\" (UniqueName: \"kubernetes.io/projected/daa7327d-1d16-4d4a-b8ca-bf1e86d0af98-kube-api-access-fqkfq\") pod \"payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb\" (UID: \"daa7327d-1d16-4d4a-b8ca-bf1e86d0af98\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb"
Sep 07 04:09:56.863385 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:56.863345    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb"
Sep 07 04:09:57.135379 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:57.135345    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb"]
Sep 07 04:09:57.138602 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:09:57.138574    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaa7327d_1d16_4d4a_b8ca_bf1e86d0af98.slice/crio-384a5c153a184ebe873c2bf0e9b5469928a83a0e3171af9a531b78562a188b04 WatchSource:0}: Error finding container 384a5c153a184ebe873c2bf0e9b5469928a83a0e3171af9a531b78562a188b04: Status 404 returned error can't find the container with id 384a5c153a184ebe873c2bf0e9b5469928a83a0e3171af9a531b78562a188b04
Sep 07 04:09:57.898081 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:57.898045    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb" event={"ID":"daa7327d-1d16-4d4a-b8ca-bf1e86d0af98","Type":"ContainerStarted","Data":"0908ca4c404d8e994af8f91e5622addf562ef307471882c9bdd1b5503900a1af"}
Sep 07 04:09:57.898290 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:57.898097    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb" event={"ID":"daa7327d-1d16-4d4a-b8ca-bf1e86d0af98","Type":"ContainerStarted","Data":"384a5c153a184ebe873c2bf0e9b5469928a83a0e3171af9a531b78562a188b04"}
Sep 07 04:09:57.898290 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:57.898119    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb"
Sep 07 04:09:57.913561 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:09:57.913512    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb" podStartSLOduration=1.9134969 podStartE2EDuration="1.9134969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:09:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:09:57.912374157 +0000 UTC m=+1284.845854875" watchObservedRunningTime="2026-09-07 04:09:57.9134969 +0000 UTC m=+1284.846977618"
Sep 07 04:10:05.790903 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:05.790857    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:10:08.905450 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:08.905422    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb"
Sep 07 04:10:16.789428 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:16.789391    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:10:18.275699 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:18.275649    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:10:19.265066 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:19.265022    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"]
Sep 07 04:10:19.278443 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:19.278415    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:10:19.773623 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:19.773575    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:10:20.188777 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:20.188748    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:10:20.478389 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:20.478286    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:10:20.671056 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:20.670998    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:10:23.975977 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:23.975943    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:10:24.075376 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:24.075344    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:10:25.077607 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.077562    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"]
Sep 07 04:10:25.081906 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.081880    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"
Sep 07 04:10:25.084288 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.084266    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-cf1797-serving-cert\""
Sep 07 04:10:25.093030 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.093000    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"]
Sep 07 04:10:25.111157 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.111122    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:10:25.211929 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.211887    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:10:25.220236 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.220171    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b57da335-bd2e-4350-a62d-3e1598b45feb-metrics-certs\") pod \"maas-api-e2e-api-b-cf1797-f464bc744-hr5mc\" (UID: \"b57da335-bd2e-4350-a62d-3e1598b45feb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"
Sep 07 04:10:25.220413 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.220308    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b57da335-bd2e-4350-a62d-3e1598b45feb-maas-api-tls\") pod \"maas-api-e2e-api-b-cf1797-f464bc744-hr5mc\" (UID: \"b57da335-bd2e-4350-a62d-3e1598b45feb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"
Sep 07 04:10:25.220462 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.220420    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b57da335-bd2e-4350-a62d-3e1598b45feb-openshift-service-ca\") pod \"maas-api-e2e-api-b-cf1797-f464bc744-hr5mc\" (UID: \"b57da335-bd2e-4350-a62d-3e1598b45feb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"
Sep 07 04:10:25.220514 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.220481    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mx67\" (UniqueName: \"kubernetes.io/projected/b57da335-bd2e-4350-a62d-3e1598b45feb-kube-api-access-6mx67\") pod \"maas-api-e2e-api-b-cf1797-f464bc744-hr5mc\" (UID: \"b57da335-bd2e-4350-a62d-3e1598b45feb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"
Sep 07 04:10:25.321358 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.321327    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b57da335-bd2e-4350-a62d-3e1598b45feb-maas-api-tls\") pod \"maas-api-e2e-api-b-cf1797-f464bc744-hr5mc\" (UID: \"b57da335-bd2e-4350-a62d-3e1598b45feb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"
Sep 07 04:10:25.321581 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.321428    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b57da335-bd2e-4350-a62d-3e1598b45feb-openshift-service-ca\") pod \"maas-api-e2e-api-b-cf1797-f464bc744-hr5mc\" (UID: \"b57da335-bd2e-4350-a62d-3e1598b45feb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"
Sep 07 04:10:25.321581 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.321487    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mx67\" (UniqueName: \"kubernetes.io/projected/b57da335-bd2e-4350-a62d-3e1598b45feb-kube-api-access-6mx67\") pod \"maas-api-e2e-api-b-cf1797-f464bc744-hr5mc\" (UID: \"b57da335-bd2e-4350-a62d-3e1598b45feb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"
Sep 07 04:10:25.321581 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:10:25.321498    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-serving-cert: secret "maas-api-e2e-api-b-cf1797-serving-cert" not found
Sep 07 04:10:25.321581 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.321520    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b57da335-bd2e-4350-a62d-3e1598b45feb-metrics-certs\") pod \"maas-api-e2e-api-b-cf1797-f464bc744-hr5mc\" (UID: \"b57da335-bd2e-4350-a62d-3e1598b45feb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"
Sep 07 04:10:25.321581 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:10:25.321573    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b57da335-bd2e-4350-a62d-3e1598b45feb-maas-api-tls podName:b57da335-bd2e-4350-a62d-3e1598b45feb nodeName:}" failed. No retries permitted until 2026-09-07 04:10:25.821552174 +0000 UTC m=+1312.755032884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b57da335-bd2e-4350-a62d-3e1598b45feb-maas-api-tls") pod "maas-api-e2e-api-b-cf1797-f464bc744-hr5mc" (UID: "b57da335-bd2e-4350-a62d-3e1598b45feb") : secret "maas-api-e2e-api-b-cf1797-serving-cert" not found
Sep 07 04:10:25.322176 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.322150    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b57da335-bd2e-4350-a62d-3e1598b45feb-openshift-service-ca\") pod \"maas-api-e2e-api-b-cf1797-f464bc744-hr5mc\" (UID: \"b57da335-bd2e-4350-a62d-3e1598b45feb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"
Sep 07 04:10:25.324119 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.324090    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b57da335-bd2e-4350-a62d-3e1598b45feb-metrics-certs\") pod \"maas-api-e2e-api-b-cf1797-f464bc744-hr5mc\" (UID: \"b57da335-bd2e-4350-a62d-3e1598b45feb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"
Sep 07 04:10:25.330613 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.330550    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mx67\" (UniqueName: \"kubernetes.io/projected/b57da335-bd2e-4350-a62d-3e1598b45feb-kube-api-access-6mx67\") pod \"maas-api-e2e-api-b-cf1797-f464bc744-hr5mc\" (UID: \"b57da335-bd2e-4350-a62d-3e1598b45feb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"
Sep 07 04:10:25.568625 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.568575    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:10:25.659750 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.659658    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:10:25.663649 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.663622    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks"]
Sep 07 04:10:25.667324 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.667304    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks"
Sep 07 04:10:25.669616 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.669596    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-cf1797\""
Sep 07 04:10:25.669616 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.669609    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-cf1797-dockercfg-xhtj7\""
Sep 07 04:10:25.681842 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.681819    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks"]
Sep 07 04:10:25.725286 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.725242    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdf8eea2-7026-4e38-acbb-c0aa3efdbe26-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks\" (UID: \"cdf8eea2-7026-4e38-acbb-c0aa3efdbe26\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks"
Sep 07 04:10:25.725457 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.725400    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bbdp\" (UniqueName: \"kubernetes.io/projected/cdf8eea2-7026-4e38-acbb-c0aa3efdbe26-kube-api-access-8bbdp\") pod \"payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks\" (UID: \"cdf8eea2-7026-4e38-acbb-c0aa3efdbe26\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks"
Sep 07 04:10:25.772818 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.772787    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:10:25.826279 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.826246    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bbdp\" (UniqueName: \"kubernetes.io/projected/cdf8eea2-7026-4e38-acbb-c0aa3efdbe26-kube-api-access-8bbdp\") pod \"payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks\" (UID: \"cdf8eea2-7026-4e38-acbb-c0aa3efdbe26\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks"
Sep 07 04:10:25.826477 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.826302    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b57da335-bd2e-4350-a62d-3e1598b45feb-maas-api-tls\") pod \"maas-api-e2e-api-b-cf1797-f464bc744-hr5mc\" (UID: \"b57da335-bd2e-4350-a62d-3e1598b45feb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"
Sep 07 04:10:25.826477 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.826332    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdf8eea2-7026-4e38-acbb-c0aa3efdbe26-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks\" (UID: \"cdf8eea2-7026-4e38-acbb-c0aa3efdbe26\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks"
Sep 07 04:10:25.827079 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.827000    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdf8eea2-7026-4e38-acbb-c0aa3efdbe26-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks\" (UID: \"cdf8eea2-7026-4e38-acbb-c0aa3efdbe26\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks"
Sep 07 04:10:25.828872 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.828850    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b57da335-bd2e-4350-a62d-3e1598b45feb-maas-api-tls\") pod \"maas-api-e2e-api-b-cf1797-f464bc744-hr5mc\" (UID: \"b57da335-bd2e-4350-a62d-3e1598b45feb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"
Sep 07 04:10:25.833841 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.833816    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bbdp\" (UniqueName: \"kubernetes.io/projected/cdf8eea2-7026-4e38-acbb-c0aa3efdbe26-kube-api-access-8bbdp\") pod \"payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks\" (UID: \"cdf8eea2-7026-4e38-acbb-c0aa3efdbe26\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks"
Sep 07 04:10:25.978630 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.978585    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks"
Sep 07 04:10:25.996892 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:25.996864    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"
Sep 07 04:10:26.254289 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:26.254264    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks"]
Sep 07 04:10:26.256381 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:10:26.256348    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdf8eea2_7026_4e38_acbb_c0aa3efdbe26.slice/crio-29c5a9db4ce4f6eb09978606851611b2bb239e1bd06065a07621b2a543b51d3f WatchSource:0}: Error finding container 29c5a9db4ce4f6eb09978606851611b2bb239e1bd06065a07621b2a543b51d3f: Status 404 returned error can't find the container with id 29c5a9db4ce4f6eb09978606851611b2bb239e1bd06065a07621b2a543b51d3f
Sep 07 04:10:26.285489 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:26.285466    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"]
Sep 07 04:10:26.290422 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:10:26.290397    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb57da335_bd2e_4350_a62d_3e1598b45feb.slice/crio-093308fec05bcf1b96d7c1c66f5d9c46178049eafe373d7b9e15fe68c65eef49 WatchSource:0}: Error finding container 093308fec05bcf1b96d7c1c66f5d9c46178049eafe373d7b9e15fe68c65eef49: Status 404 returned error can't find the container with id 093308fec05bcf1b96d7c1c66f5d9c46178049eafe373d7b9e15fe68c65eef49
Sep 07 04:10:27.056937 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:27.056893    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks" event={"ID":"cdf8eea2-7026-4e38-acbb-c0aa3efdbe26","Type":"ContainerStarted","Data":"02a7c0f52f96a885b542925d4c36031feac50ddbe6e6158be126b87feb7b8dd4"}
Sep 07 04:10:27.057186 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:27.056957    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks" event={"ID":"cdf8eea2-7026-4e38-acbb-c0aa3efdbe26","Type":"ContainerStarted","Data":"29c5a9db4ce4f6eb09978606851611b2bb239e1bd06065a07621b2a543b51d3f"}
Sep 07 04:10:27.057186 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:27.056987    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks"
Sep 07 04:10:27.058459 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:27.058435    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc" event={"ID":"b57da335-bd2e-4350-a62d-3e1598b45feb","Type":"ContainerStarted","Data":"1740e296f28dc58dbe55d1d944c667c6b2ff1c209ebc0f40d375c6b6045aa5e0"}
Sep 07 04:10:27.058553 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:27.058482    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc" event={"ID":"b57da335-bd2e-4350-a62d-3e1598b45feb","Type":"ContainerStarted","Data":"093308fec05bcf1b96d7c1c66f5d9c46178049eafe373d7b9e15fe68c65eef49"}
Sep 07 04:10:27.058553 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:27.058503    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"
Sep 07 04:10:27.073265 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:27.073197    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks" podStartSLOduration=2.073180541 podStartE2EDuration="2.073180541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:10:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:10:27.070861722 +0000 UTC m=+1314.004342442" watchObservedRunningTime="2026-09-07 04:10:27.073180541 +0000 UTC m=+1314.006661315"
Sep 07 04:10:27.087955 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:27.087894    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc" podStartSLOduration=1.743898336 podStartE2EDuration="2.087875882s" totalImagesPullingTime="343.977546ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:10:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:10:27.085958982 +0000 UTC m=+1314.019439701" watchObservedRunningTime="2026-09-07 04:10:27.087875882 +0000 UTC m=+1314.021356651"
Sep 07 04:10:27.794347 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:27.794316    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:10:27.895017 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:27.894982    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:10:33.068906 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:33.068877    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"
Sep 07 04:10:34.286416 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:34.286360    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq" podUID="d718aa52-da42-417f-b62a-a3ca55cf99cb" containerName="main" containerID="cri-o://eb8e95604450fa152da21bce985f3d5101784258e61ae01a89416b1976d9c549" gracePeriod=45
Sep 07 04:10:34.407521 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:34.407481    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:10:34.544476 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:34.544408    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:10:34.713544 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:34.713514    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d718aa52-da42-417f-b62a-a3ca55cf99cb-tls-certs\") pod \"d718aa52-da42-417f-b62a-a3ca55cf99cb\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") "
Sep 07 04:10:34.713742 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:34.713618    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-model-cache\") pod \"d718aa52-da42-417f-b62a-a3ca55cf99cb\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") "
Sep 07 04:10:34.713742 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:34.713673    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-dshm\") pod \"d718aa52-da42-417f-b62a-a3ca55cf99cb\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") "
Sep 07 04:10:34.713742 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:34.713702    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-home\") pod \"d718aa52-da42-417f-b62a-a3ca55cf99cb\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") "
Sep 07 04:10:34.713742 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:34.713728    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvbzr\" (UniqueName: \"kubernetes.io/projected/d718aa52-da42-417f-b62a-a3ca55cf99cb-kube-api-access-gvbzr\") pod \"d718aa52-da42-417f-b62a-a3ca55cf99cb\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") "
Sep 07 04:10:34.714025 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:34.713793    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-tmp-dir\") pod \"d718aa52-da42-417f-b62a-a3ca55cf99cb\" (UID: \"d718aa52-da42-417f-b62a-a3ca55cf99cb\") "
Sep 07 04:10:34.714025 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:34.713903    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-model-cache" (OuterVolumeSpecName: "model-cache") pod "d718aa52-da42-417f-b62a-a3ca55cf99cb" (UID: "d718aa52-da42-417f-b62a-a3ca55cf99cb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:10:34.714025 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:34.713915    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-home" (OuterVolumeSpecName: "home") pod "d718aa52-da42-417f-b62a-a3ca55cf99cb" (UID: "d718aa52-da42-417f-b62a-a3ca55cf99cb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:10:34.714188 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:34.714149    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d718aa52-da42-417f-b62a-a3ca55cf99cb" (UID: "d718aa52-da42-417f-b62a-a3ca55cf99cb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:10:34.714188 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:34.714175    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-model-cache\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:10:34.714292 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:34.714191    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-home\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:10:34.716188 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:34.716165    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-dshm" (OuterVolumeSpecName: "dshm") pod "d718aa52-da42-417f-b62a-a3ca55cf99cb" (UID: "d718aa52-da42-417f-b62a-a3ca55cf99cb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:10:34.716327 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:34.716186    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d718aa52-da42-417f-b62a-a3ca55cf99cb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d718aa52-da42-417f-b62a-a3ca55cf99cb" (UID: "d718aa52-da42-417f-b62a-a3ca55cf99cb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:10:34.716327 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:34.716296    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d718aa52-da42-417f-b62a-a3ca55cf99cb-kube-api-access-gvbzr" (OuterVolumeSpecName: "kube-api-access-gvbzr") pod "d718aa52-da42-417f-b62a-a3ca55cf99cb" (UID: "d718aa52-da42-417f-b62a-a3ca55cf99cb"). InnerVolumeSpecName "kube-api-access-gvbzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:10:34.815259 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:34.815163    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-tmp-dir\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:10:34.815259 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:34.815191    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d718aa52-da42-417f-b62a-a3ca55cf99cb-tls-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:10:34.815259 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:34.815200    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d718aa52-da42-417f-b62a-a3ca55cf99cb-dshm\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:10:34.815259 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:34.815221    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvbzr\" (UniqueName: \"kubernetes.io/projected/d718aa52-da42-417f-b62a-a3ca55cf99cb-kube-api-access-gvbzr\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:10:35.101942 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:35.101852    2583 generic.go:356] "Generic (PLEG): container finished" podID="d718aa52-da42-417f-b62a-a3ca55cf99cb" containerID="eb8e95604450fa152da21bce985f3d5101784258e61ae01a89416b1976d9c549" exitCode=0
Sep 07 04:10:35.101942 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:35.101893    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"
Sep 07 04:10:35.102122 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:35.101939    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq" event={"ID":"d718aa52-da42-417f-b62a-a3ca55cf99cb","Type":"ContainerDied","Data":"eb8e95604450fa152da21bce985f3d5101784258e61ae01a89416b1976d9c549"}
Sep 07 04:10:35.102122 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:35.101989    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq" event={"ID":"d718aa52-da42-417f-b62a-a3ca55cf99cb","Type":"ContainerDied","Data":"3e1de1e3f17a5eeff9246641090de9f911e8577010d1c375dad13069d8dafb4a"}
Sep 07 04:10:35.102122 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:35.102010    2583 scope.go:117] "RemoveContainer" containerID="eb8e95604450fa152da21bce985f3d5101784258e61ae01a89416b1976d9c549"
Sep 07 04:10:35.111779 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:35.111748    2583 scope.go:117] "RemoveContainer" containerID="eb8e95604450fa152da21bce985f3d5101784258e61ae01a89416b1976d9c549"
Sep 07 04:10:35.112064 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:10:35.112044    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb8e95604450fa152da21bce985f3d5101784258e61ae01a89416b1976d9c549\": container with ID starting with eb8e95604450fa152da21bce985f3d5101784258e61ae01a89416b1976d9c549 not found: ID does not exist" containerID="eb8e95604450fa152da21bce985f3d5101784258e61ae01a89416b1976d9c549"
Sep 07 04:10:35.112137 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:35.112072    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb8e95604450fa152da21bce985f3d5101784258e61ae01a89416b1976d9c549"} err="failed to get container status \"eb8e95604450fa152da21bce985f3d5101784258e61ae01a89416b1976d9c549\": rpc error: code = NotFound desc = could not find container \"eb8e95604450fa152da21bce985f3d5101784258e61ae01a89416b1976d9c549\": container with ID starting with eb8e95604450fa152da21bce985f3d5101784258e61ae01a89416b1976d9c549 not found: ID does not exist"
Sep 07 04:10:35.121407 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:35.121380    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"]
Sep 07 04:10:35.125810 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:35.125787    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-434d57/sub-test-model-434d57-kserve-7fd6484df-dxhhq"]
Sep 07 04:10:35.524508 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:35.524475    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d718aa52-da42-417f-b62a-a3ca55cf99cb" path="/var/lib/kubelet/pods/d718aa52-da42-417f-b62a-a3ca55cf99cb/volumes"
Sep 07 04:10:38.066328 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:38.066288    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks"
Sep 07 04:10:45.912293 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:45.912246    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:10:48.883606 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:48.883569    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:10:50.482827 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.482795    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"]
Sep 07 04:10:50.483440 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.483413    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d718aa52-da42-417f-b62a-a3ca55cf99cb" containerName="main"
Sep 07 04:10:50.483440 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.483435    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d718aa52-da42-417f-b62a-a3ca55cf99cb" containerName="main"
Sep 07 04:10:50.483605 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.483558    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d718aa52-da42-417f-b62a-a3ca55cf99cb" containerName="main"
Sep 07 04:10:50.488477 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.488456    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:10:50.491047 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.491027    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-b8cea6\"/\"kube-root-ca.crt\""
Sep 07 04:10:50.492226 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.492190    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-b8cea6\"/\"openshift-service-ca.crt\""
Sep 07 04:10:50.492365 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.492324    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-b8cea6\"/\"rate-test-model-b8cea6-kserve-self-signed-certs\""
Sep 07 04:10:50.492463 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.492443    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-b8cea6\"/\"default-dockercfg-vxkmj\""
Sep 07 04:10:50.494911 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.494878    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"]
Sep 07 04:10:50.506253 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.506206    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:10:50.563332 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.563294    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-model-cache\") pod \"rate-test-model-b8cea6-kserve-f9b6d696c-89jh2\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:10:50.563534 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.563362    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/697c4957-ab95-41c1-9f44-fa5c125cb7a0-tls-certs\") pod \"rate-test-model-b8cea6-kserve-f9b6d696c-89jh2\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:10:50.563534 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.563432    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-home\") pod \"rate-test-model-b8cea6-kserve-f9b6d696c-89jh2\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:10:50.563534 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.563465    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-tmp-dir\") pod \"rate-test-model-b8cea6-kserve-f9b6d696c-89jh2\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:10:50.563534 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.563485    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-dshm\") pod \"rate-test-model-b8cea6-kserve-f9b6d696c-89jh2\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:10:50.563534 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.563513    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krgwc\" (UniqueName: \"kubernetes.io/projected/697c4957-ab95-41c1-9f44-fa5c125cb7a0-kube-api-access-krgwc\") pod \"rate-test-model-b8cea6-kserve-f9b6d696c-89jh2\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:10:50.664026 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.663989    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-model-cache\") pod \"rate-test-model-b8cea6-kserve-f9b6d696c-89jh2\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:10:50.664201 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.664035    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/697c4957-ab95-41c1-9f44-fa5c125cb7a0-tls-certs\") pod \"rate-test-model-b8cea6-kserve-f9b6d696c-89jh2\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:10:50.664201 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.664059    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-home\") pod \"rate-test-model-b8cea6-kserve-f9b6d696c-89jh2\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:10:50.664201 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.664073    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-tmp-dir\") pod \"rate-test-model-b8cea6-kserve-f9b6d696c-89jh2\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:10:50.664201 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.664191    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-dshm\") pod \"rate-test-model-b8cea6-kserve-f9b6d696c-89jh2\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:10:50.664485 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.664274    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krgwc\" (UniqueName: \"kubernetes.io/projected/697c4957-ab95-41c1-9f44-fa5c125cb7a0-kube-api-access-krgwc\") pod \"rate-test-model-b8cea6-kserve-f9b6d696c-89jh2\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:10:50.664485 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.664467    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-home\") pod \"rate-test-model-b8cea6-kserve-f9b6d696c-89jh2\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:10:50.664589 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.664524    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-model-cache\") pod \"rate-test-model-b8cea6-kserve-f9b6d696c-89jh2\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:10:50.664589 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.664534    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-tmp-dir\") pod \"rate-test-model-b8cea6-kserve-f9b6d696c-89jh2\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:10:50.666553 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.666533    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-dshm\") pod \"rate-test-model-b8cea6-kserve-f9b6d696c-89jh2\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:10:50.666723 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.666706    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/697c4957-ab95-41c1-9f44-fa5c125cb7a0-tls-certs\") pod \"rate-test-model-b8cea6-kserve-f9b6d696c-89jh2\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:10:50.672233 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.672198    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krgwc\" (UniqueName: \"kubernetes.io/projected/697c4957-ab95-41c1-9f44-fa5c125cb7a0-kube-api-access-krgwc\") pod \"rate-test-model-b8cea6-kserve-f9b6d696c-89jh2\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") " pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:10:50.799994 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:50.799896    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:10:51.077518 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:51.077490    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"]
Sep 07 04:10:51.079382 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:10:51.079345    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod697c4957_ab95_41c1_9f44_fa5c125cb7a0.slice/crio-2bb10b6ec575c05a74706286b187811feeec7653e63f47cea7b1b3dabad316e7 WatchSource:0}: Error finding container 2bb10b6ec575c05a74706286b187811feeec7653e63f47cea7b1b3dabad316e7: Status 404 returned error can't find the container with id 2bb10b6ec575c05a74706286b187811feeec7653e63f47cea7b1b3dabad316e7
Sep 07 04:10:51.186911 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:51.186879    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2" event={"ID":"697c4957-ab95-41c1-9f44-fa5c125cb7a0","Type":"ContainerStarted","Data":"64fca9beccbe5629469aa695228b2eafe8c9fcd096ac8dc8bf47a45928e2fe06"}
Sep 07 04:10:51.187082 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:51.186921    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2" event={"ID":"697c4957-ab95-41c1-9f44-fa5c125cb7a0","Type":"ContainerStarted","Data":"2bb10b6ec575c05a74706286b187811feeec7653e63f47cea7b1b3dabad316e7"}
Sep 07 04:10:51.203825 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:51.203769    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2" podStartSLOduration=1.203752243 podStartE2EDuration="1.203752243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:10:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:10:51.202198569 +0000 UTC m=+1338.135679284" watchObservedRunningTime="2026-09-07 04:10:51.203752243 +0000 UTC m=+1338.137232960"
Sep 07 04:10:51.916082 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:10:51.916052    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:00.800356 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:00.800259    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:11:00.800356 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:00.800324    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:11:00.812963 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:00.812931    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:11:01.256724 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:01.256698    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:11:04.412666 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:04.412633    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:07.200939 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:07.200897    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:07.230163 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:07.230133    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb"]
Sep 07 04:11:07.230416 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:07.230385    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb" podUID="daa7327d-1d16-4d4a-b8ca-bf1e86d0af98" containerName="payload-pre-processing" containerID="cri-o://0908ca4c404d8e994af8f91e5622addf562ef307471882c9bdd1b5503900a1af" gracePeriod=30
Sep 07 04:11:07.262487 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:07.262455    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:07.362756 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:07.362702    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:07.417965 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:07.417923    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:07.492671 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:07.492648    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb"
Sep 07 04:11:07.630148 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:07.630110    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daa7327d-1d16-4d4a-b8ca-bf1e86d0af98-plugins-config-volume\") pod \"daa7327d-1d16-4d4a-b8ca-bf1e86d0af98\" (UID: \"daa7327d-1d16-4d4a-b8ca-bf1e86d0af98\") "
Sep 07 04:11:07.630353 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:07.630178    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqkfq\" (UniqueName: \"kubernetes.io/projected/daa7327d-1d16-4d4a-b8ca-bf1e86d0af98-kube-api-access-fqkfq\") pod \"daa7327d-1d16-4d4a-b8ca-bf1e86d0af98\" (UID: \"daa7327d-1d16-4d4a-b8ca-bf1e86d0af98\") "
Sep 07 04:11:07.630627 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:07.630596    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/daa7327d-1d16-4d4a-b8ca-bf1e86d0af98-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "daa7327d-1d16-4d4a-b8ca-bf1e86d0af98" (UID: "daa7327d-1d16-4d4a-b8ca-bf1e86d0af98"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 04:11:07.632510 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:07.632483    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daa7327d-1d16-4d4a-b8ca-bf1e86d0af98-kube-api-access-fqkfq" (OuterVolumeSpecName: "kube-api-access-fqkfq") pod "daa7327d-1d16-4d4a-b8ca-bf1e86d0af98" (UID: "daa7327d-1d16-4d4a-b8ca-bf1e86d0af98"). InnerVolumeSpecName "kube-api-access-fqkfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:11:07.732010 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:07.731926    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daa7327d-1d16-4d4a-b8ca-bf1e86d0af98-plugins-config-volume\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:11:07.732010 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:07.731957    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqkfq\" (UniqueName: \"kubernetes.io/projected/daa7327d-1d16-4d4a-b8ca-bf1e86d0af98-kube-api-access-fqkfq\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:11:07.760334 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:07.760300    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:08.283515 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:08.283478    2583 generic.go:356] "Generic (PLEG): container finished" podID="daa7327d-1d16-4d4a-b8ca-bf1e86d0af98" containerID="0908ca4c404d8e994af8f91e5622addf562ef307471882c9bdd1b5503900a1af" exitCode=0
Sep 07 04:11:08.284032 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:08.283522    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb"
Sep 07 04:11:08.284032 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:08.283579    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb" event={"ID":"daa7327d-1d16-4d4a-b8ca-bf1e86d0af98","Type":"ContainerDied","Data":"0908ca4c404d8e994af8f91e5622addf562ef307471882c9bdd1b5503900a1af"}
Sep 07 04:11:08.284032 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:08.283641    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb" event={"ID":"daa7327d-1d16-4d4a-b8ca-bf1e86d0af98","Type":"ContainerDied","Data":"384a5c153a184ebe873c2bf0e9b5469928a83a0e3171af9a531b78562a188b04"}
Sep 07 04:11:08.284032 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:08.283679    2583 scope.go:117] "RemoveContainer" containerID="0908ca4c404d8e994af8f91e5622addf562ef307471882c9bdd1b5503900a1af"
Sep 07 04:11:08.293519 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:08.293495    2583 scope.go:117] "RemoveContainer" containerID="0908ca4c404d8e994af8f91e5622addf562ef307471882c9bdd1b5503900a1af"
Sep 07 04:11:08.293917 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:11:08.293825    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0908ca4c404d8e994af8f91e5622addf562ef307471882c9bdd1b5503900a1af\": container with ID starting with 0908ca4c404d8e994af8f91e5622addf562ef307471882c9bdd1b5503900a1af not found: ID does not exist" containerID="0908ca4c404d8e994af8f91e5622addf562ef307471882c9bdd1b5503900a1af"
Sep 07 04:11:08.294008 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:08.293926    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0908ca4c404d8e994af8f91e5622addf562ef307471882c9bdd1b5503900a1af"} err="failed to get container status \"0908ca4c404d8e994af8f91e5622addf562ef307471882c9bdd1b5503900a1af\": rpc error: code = NotFound desc = could not find container \"0908ca4c404d8e994af8f91e5622addf562ef307471882c9bdd1b5503900a1af\": container with ID starting with 0908ca4c404d8e994af8f91e5622addf562ef307471882c9bdd1b5503900a1af not found: ID does not exist"
Sep 07 04:11:08.304949 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:08.304908    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb"]
Sep 07 04:11:08.306883 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:08.306850    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-467e44-77c8c9b554-wfffb"]
Sep 07 04:11:09.522639 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:09.522603    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="daa7327d-1d16-4d4a-b8ca-bf1e86d0af98" path="/var/lib/kubelet/pods/daa7327d-1d16-4d4a-b8ca-bf1e86d0af98/volumes"
Sep 07 04:11:13.455755 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:13.455718    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:13.520992 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:13.520953    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:13.716611 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:13.716534    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:14.303688 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:14.303634    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:16.786894 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:16.786835    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:29.434547 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.434508    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"]
Sep 07 04:11:29.435133 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.434817    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc" podUID="b57da335-bd2e-4350-a62d-3e1598b45feb" containerName="maas-api" containerID="cri-o://1740e296f28dc58dbe55d1d944c667c6b2ff1c209ebc0f40d375c6b6045aa5e0" gracePeriod=30
Sep 07 04:11:29.459818 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.459789    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:29.490774 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.490738    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks"]
Sep 07 04:11:29.491069 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.491035    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks" podUID="cdf8eea2-7026-4e38-acbb-c0aa3efdbe26" containerName="payload-pre-processing" containerID="cri-o://02a7c0f52f96a885b542925d4c36031feac50ddbe6e6158be126b87feb7b8dd4" gracePeriod=30
Sep 07 04:11:29.559646 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.559618    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:29.620699 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.620661    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:29.677778 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.677726    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:29.742140 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.742119    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"
Sep 07 04:11:29.774289 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.774265    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks"
Sep 07 04:11:29.844054 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.844025    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b57da335-bd2e-4350-a62d-3e1598b45feb-maas-api-tls\") pod \"b57da335-bd2e-4350-a62d-3e1598b45feb\" (UID: \"b57da335-bd2e-4350-a62d-3e1598b45feb\") "
Sep 07 04:11:29.844054 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.844054    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdf8eea2-7026-4e38-acbb-c0aa3efdbe26-plugins-config-volume\") pod \"cdf8eea2-7026-4e38-acbb-c0aa3efdbe26\" (UID: \"cdf8eea2-7026-4e38-acbb-c0aa3efdbe26\") "
Sep 07 04:11:29.844302 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.844110    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b57da335-bd2e-4350-a62d-3e1598b45feb-openshift-service-ca\") pod \"b57da335-bd2e-4350-a62d-3e1598b45feb\" (UID: \"b57da335-bd2e-4350-a62d-3e1598b45feb\") "
Sep 07 04:11:29.844302 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.844165    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b57da335-bd2e-4350-a62d-3e1598b45feb-metrics-certs\") pod \"b57da335-bd2e-4350-a62d-3e1598b45feb\" (UID: \"b57da335-bd2e-4350-a62d-3e1598b45feb\") "
Sep 07 04:11:29.844302 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.844258    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mx67\" (UniqueName: \"kubernetes.io/projected/b57da335-bd2e-4350-a62d-3e1598b45feb-kube-api-access-6mx67\") pod \"b57da335-bd2e-4350-a62d-3e1598b45feb\" (UID: \"b57da335-bd2e-4350-a62d-3e1598b45feb\") "
Sep 07 04:11:29.844474 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.844309    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bbdp\" (UniqueName: \"kubernetes.io/projected/cdf8eea2-7026-4e38-acbb-c0aa3efdbe26-kube-api-access-8bbdp\") pod \"cdf8eea2-7026-4e38-acbb-c0aa3efdbe26\" (UID: \"cdf8eea2-7026-4e38-acbb-c0aa3efdbe26\") "
Sep 07 04:11:29.844551 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.844511    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b57da335-bd2e-4350-a62d-3e1598b45feb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b57da335-bd2e-4350-a62d-3e1598b45feb" (UID: "b57da335-bd2e-4350-a62d-3e1598b45feb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 04:11:29.844551 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.844530    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdf8eea2-7026-4e38-acbb-c0aa3efdbe26-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cdf8eea2-7026-4e38-acbb-c0aa3efdbe26" (UID: "cdf8eea2-7026-4e38-acbb-c0aa3efdbe26"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 04:11:29.844673 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.844637    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdf8eea2-7026-4e38-acbb-c0aa3efdbe26-plugins-config-volume\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:11:29.844673 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.844657    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b57da335-bd2e-4350-a62d-3e1598b45feb-openshift-service-ca\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:11:29.846483 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.846456    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b57da335-bd2e-4350-a62d-3e1598b45feb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b57da335-bd2e-4350-a62d-3e1598b45feb" (UID: "b57da335-bd2e-4350-a62d-3e1598b45feb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:11:29.846618 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.846518    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b57da335-bd2e-4350-a62d-3e1598b45feb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b57da335-bd2e-4350-a62d-3e1598b45feb" (UID: "b57da335-bd2e-4350-a62d-3e1598b45feb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:11:29.846618 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.846537    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b57da335-bd2e-4350-a62d-3e1598b45feb-kube-api-access-6mx67" (OuterVolumeSpecName: "kube-api-access-6mx67") pod "b57da335-bd2e-4350-a62d-3e1598b45feb" (UID: "b57da335-bd2e-4350-a62d-3e1598b45feb"). InnerVolumeSpecName "kube-api-access-6mx67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:11:29.846618 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.846567    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdf8eea2-7026-4e38-acbb-c0aa3efdbe26-kube-api-access-8bbdp" (OuterVolumeSpecName: "kube-api-access-8bbdp") pod "cdf8eea2-7026-4e38-acbb-c0aa3efdbe26" (UID: "cdf8eea2-7026-4e38-acbb-c0aa3efdbe26"). InnerVolumeSpecName "kube-api-access-8bbdp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:11:29.946102 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.946065    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mx67\" (UniqueName: \"kubernetes.io/projected/b57da335-bd2e-4350-a62d-3e1598b45feb-kube-api-access-6mx67\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:11:29.946102 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.946096    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bbdp\" (UniqueName: \"kubernetes.io/projected/cdf8eea2-7026-4e38-acbb-c0aa3efdbe26-kube-api-access-8bbdp\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:11:29.946102 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.946106    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b57da335-bd2e-4350-a62d-3e1598b45feb-maas-api-tls\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:11:29.946348 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.946117    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b57da335-bd2e-4350-a62d-3e1598b45feb-metrics-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:11:29.977158 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:29.977072    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:30.414285 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:30.414184    2583 generic.go:356] "Generic (PLEG): container finished" podID="b57da335-bd2e-4350-a62d-3e1598b45feb" containerID="1740e296f28dc58dbe55d1d944c667c6b2ff1c209ebc0f40d375c6b6045aa5e0" exitCode=0
Sep 07 04:11:30.414285 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:30.414260    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"
Sep 07 04:11:30.414508 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:30.414255    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc" event={"ID":"b57da335-bd2e-4350-a62d-3e1598b45feb","Type":"ContainerDied","Data":"1740e296f28dc58dbe55d1d944c667c6b2ff1c209ebc0f40d375c6b6045aa5e0"}
Sep 07 04:11:30.414508 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:30.414403    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc" event={"ID":"b57da335-bd2e-4350-a62d-3e1598b45feb","Type":"ContainerDied","Data":"093308fec05bcf1b96d7c1c66f5d9c46178049eafe373d7b9e15fe68c65eef49"}
Sep 07 04:11:30.414508 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:30.414431    2583 scope.go:117] "RemoveContainer" containerID="1740e296f28dc58dbe55d1d944c667c6b2ff1c209ebc0f40d375c6b6045aa5e0"
Sep 07 04:11:30.416227 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:30.416185    2583 generic.go:356] "Generic (PLEG): container finished" podID="cdf8eea2-7026-4e38-acbb-c0aa3efdbe26" containerID="02a7c0f52f96a885b542925d4c36031feac50ddbe6e6158be126b87feb7b8dd4" exitCode=0
Sep 07 04:11:30.416379 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:30.416277    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks"
Sep 07 04:11:30.416379 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:30.416282    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks" event={"ID":"cdf8eea2-7026-4e38-acbb-c0aa3efdbe26","Type":"ContainerDied","Data":"02a7c0f52f96a885b542925d4c36031feac50ddbe6e6158be126b87feb7b8dd4"}
Sep 07 04:11:30.416379 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:30.416333    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks" event={"ID":"cdf8eea2-7026-4e38-acbb-c0aa3efdbe26","Type":"ContainerDied","Data":"29c5a9db4ce4f6eb09978606851611b2bb239e1bd06065a07621b2a543b51d3f"}
Sep 07 04:11:30.425451 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:30.425428    2583 scope.go:117] "RemoveContainer" containerID="1740e296f28dc58dbe55d1d944c667c6b2ff1c209ebc0f40d375c6b6045aa5e0"
Sep 07 04:11:30.425795 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:11:30.425775    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1740e296f28dc58dbe55d1d944c667c6b2ff1c209ebc0f40d375c6b6045aa5e0\": container with ID starting with 1740e296f28dc58dbe55d1d944c667c6b2ff1c209ebc0f40d375c6b6045aa5e0 not found: ID does not exist" containerID="1740e296f28dc58dbe55d1d944c667c6b2ff1c209ebc0f40d375c6b6045aa5e0"
Sep 07 04:11:30.425870 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:30.425802    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1740e296f28dc58dbe55d1d944c667c6b2ff1c209ebc0f40d375c6b6045aa5e0"} err="failed to get container status \"1740e296f28dc58dbe55d1d944c667c6b2ff1c209ebc0f40d375c6b6045aa5e0\": rpc error: code = NotFound desc = could not find container \"1740e296f28dc58dbe55d1d944c667c6b2ff1c209ebc0f40d375c6b6045aa5e0\": container with ID starting with 1740e296f28dc58dbe55d1d944c667c6b2ff1c209ebc0f40d375c6b6045aa5e0 not found: ID does not exist"
Sep 07 04:11:30.425870 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:30.425820    2583 scope.go:117] "RemoveContainer" containerID="02a7c0f52f96a885b542925d4c36031feac50ddbe6e6158be126b87feb7b8dd4"
Sep 07 04:11:30.434566 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:30.434548    2583 scope.go:117] "RemoveContainer" containerID="02a7c0f52f96a885b542925d4c36031feac50ddbe6e6158be126b87feb7b8dd4"
Sep 07 04:11:30.434842 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:11:30.434824    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02a7c0f52f96a885b542925d4c36031feac50ddbe6e6158be126b87feb7b8dd4\": container with ID starting with 02a7c0f52f96a885b542925d4c36031feac50ddbe6e6158be126b87feb7b8dd4 not found: ID does not exist" containerID="02a7c0f52f96a885b542925d4c36031feac50ddbe6e6158be126b87feb7b8dd4"
Sep 07 04:11:30.434894 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:30.434850    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02a7c0f52f96a885b542925d4c36031feac50ddbe6e6158be126b87feb7b8dd4"} err="failed to get container status \"02a7c0f52f96a885b542925d4c36031feac50ddbe6e6158be126b87feb7b8dd4\": rpc error: code = NotFound desc = could not find container \"02a7c0f52f96a885b542925d4c36031feac50ddbe6e6158be126b87feb7b8dd4\": container with ID starting with 02a7c0f52f96a885b542925d4c36031feac50ddbe6e6158be126b87feb7b8dd4 not found: ID does not exist"
Sep 07 04:11:30.440756 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:30.440691    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"]
Sep 07 04:11:30.442757 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:30.442729    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cf1797-f464bc744-hr5mc"]
Sep 07 04:11:30.454039 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:30.453642    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks"]
Sep 07 04:11:30.457066 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:30.457043    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-cf1797-69d587b498-d86ks"]
Sep 07 04:11:31.522522 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:31.522479    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b57da335-bd2e-4350-a62d-3e1598b45feb" path="/var/lib/kubelet/pods/b57da335-bd2e-4350-a62d-3e1598b45feb/volumes"
Sep 07 04:11:31.523043 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:31.523021    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdf8eea2-7026-4e38-acbb-c0aa3efdbe26" path="/var/lib/kubelet/pods/cdf8eea2-7026-4e38-acbb-c0aa3efdbe26/volumes"
Sep 07 04:11:32.627727 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:32.627690    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:32.842651 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:32.842620    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:35.736390 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:35.736330    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:35.837517 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:35.837481    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:36.003707 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:36.003609    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:36.484416 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:36.484368    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:46.716446 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:46.716400    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"]
Sep 07 04:11:46.716912 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:46.716678    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh" podUID="3fdee0a9-1427-4f4a-897d-594ea81e87b9" containerName="maas-api" containerID="cri-o://86e5fcb303ea755c9a3776bd7bcea7d724639f2e875f9e6fc7c1963ba8337acb" gracePeriod=30
Sep 07 04:11:46.731850 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:46.731814    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:46.797950 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:46.797916    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:46.861485 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:46.861450    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:46.969076 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:46.968996    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:46.979574 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:46.979551    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"
Sep 07 04:11:47.112791 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:47.112755    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcckq\" (UniqueName: \"kubernetes.io/projected/3fdee0a9-1427-4f4a-897d-594ea81e87b9-kube-api-access-tcckq\") pod \"3fdee0a9-1427-4f4a-897d-594ea81e87b9\" (UID: \"3fdee0a9-1427-4f4a-897d-594ea81e87b9\") "
Sep 07 04:11:47.112963 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:47.112848    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fdee0a9-1427-4f4a-897d-594ea81e87b9-maas-api-tls\") pod \"3fdee0a9-1427-4f4a-897d-594ea81e87b9\" (UID: \"3fdee0a9-1427-4f4a-897d-594ea81e87b9\") "
Sep 07 04:11:47.112963 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:47.112887    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fdee0a9-1427-4f4a-897d-594ea81e87b9-openshift-service-ca\") pod \"3fdee0a9-1427-4f4a-897d-594ea81e87b9\" (UID: \"3fdee0a9-1427-4f4a-897d-594ea81e87b9\") "
Sep 07 04:11:47.112963 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:47.112922    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fdee0a9-1427-4f4a-897d-594ea81e87b9-metrics-certs\") pod \"3fdee0a9-1427-4f4a-897d-594ea81e87b9\" (UID: \"3fdee0a9-1427-4f4a-897d-594ea81e87b9\") "
Sep 07 04:11:47.113385 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:47.113349    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fdee0a9-1427-4f4a-897d-594ea81e87b9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3fdee0a9-1427-4f4a-897d-594ea81e87b9" (UID: "3fdee0a9-1427-4f4a-897d-594ea81e87b9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 04:11:47.115162 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:47.115127    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fdee0a9-1427-4f4a-897d-594ea81e87b9-kube-api-access-tcckq" (OuterVolumeSpecName: "kube-api-access-tcckq") pod "3fdee0a9-1427-4f4a-897d-594ea81e87b9" (UID: "3fdee0a9-1427-4f4a-897d-594ea81e87b9"). InnerVolumeSpecName "kube-api-access-tcckq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:11:47.115328 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:47.115159    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fdee0a9-1427-4f4a-897d-594ea81e87b9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3fdee0a9-1427-4f4a-897d-594ea81e87b9" (UID: "3fdee0a9-1427-4f4a-897d-594ea81e87b9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:11:47.115499 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:47.115478    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fdee0a9-1427-4f4a-897d-594ea81e87b9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3fdee0a9-1427-4f4a-897d-594ea81e87b9" (UID: "3fdee0a9-1427-4f4a-897d-594ea81e87b9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:11:47.213813 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:47.213775    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcckq\" (UniqueName: \"kubernetes.io/projected/3fdee0a9-1427-4f4a-897d-594ea81e87b9-kube-api-access-tcckq\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:11:47.213813 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:47.213807    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fdee0a9-1427-4f4a-897d-594ea81e87b9-maas-api-tls\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:11:47.213813 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:47.213818    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fdee0a9-1427-4f4a-897d-594ea81e87b9-openshift-service-ca\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:11:47.214064 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:47.213826    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fdee0a9-1427-4f4a-897d-594ea81e87b9-metrics-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:11:47.284519 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:47.284429    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:47.507521 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:47.507482    2583 generic.go:356] "Generic (PLEG): container finished" podID="3fdee0a9-1427-4f4a-897d-594ea81e87b9" containerID="86e5fcb303ea755c9a3776bd7bcea7d724639f2e875f9e6fc7c1963ba8337acb" exitCode=0
Sep 07 04:11:47.507521 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:47.507528    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"
Sep 07 04:11:47.507780 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:47.507562    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh" event={"ID":"3fdee0a9-1427-4f4a-897d-594ea81e87b9","Type":"ContainerDied","Data":"86e5fcb303ea755c9a3776bd7bcea7d724639f2e875f9e6fc7c1963ba8337acb"}
Sep 07 04:11:47.507780 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:47.507618    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh" event={"ID":"3fdee0a9-1427-4f4a-897d-594ea81e87b9","Type":"ContainerDied","Data":"f1a0bc06baf5c3a91be8646b12b51a13e01816d02bd484903a12e1ecfce9671e"}
Sep 07 04:11:47.507780 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:47.507642    2583 scope.go:117] "RemoveContainer" containerID="86e5fcb303ea755c9a3776bd7bcea7d724639f2e875f9e6fc7c1963ba8337acb"
Sep 07 04:11:47.518239 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:47.518185    2583 scope.go:117] "RemoveContainer" containerID="86e5fcb303ea755c9a3776bd7bcea7d724639f2e875f9e6fc7c1963ba8337acb"
Sep 07 04:11:47.518540 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:11:47.518513    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86e5fcb303ea755c9a3776bd7bcea7d724639f2e875f9e6fc7c1963ba8337acb\": container with ID starting with 86e5fcb303ea755c9a3776bd7bcea7d724639f2e875f9e6fc7c1963ba8337acb not found: ID does not exist" containerID="86e5fcb303ea755c9a3776bd7bcea7d724639f2e875f9e6fc7c1963ba8337acb"
Sep 07 04:11:47.518669 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:47.518551    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86e5fcb303ea755c9a3776bd7bcea7d724639f2e875f9e6fc7c1963ba8337acb"} err="failed to get container status \"86e5fcb303ea755c9a3776bd7bcea7d724639f2e875f9e6fc7c1963ba8337acb\": rpc error: code = NotFound desc = could not find container \"86e5fcb303ea755c9a3776bd7bcea7d724639f2e875f9e6fc7c1963ba8337acb\": container with ID starting with 86e5fcb303ea755c9a3776bd7bcea7d724639f2e875f9e6fc7c1963ba8337acb not found: ID does not exist"
Sep 07 04:11:47.528999 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:47.528966    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"]
Sep 07 04:11:47.531892 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:47.531866    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-747fee-7647c6cbb7-wtmrh"]
Sep 07 04:11:49.522193 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:49.522159    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fdee0a9-1427-4f4a-897d-594ea81e87b9" path="/var/lib/kubelet/pods/3fdee0a9-1427-4f4a-897d-594ea81e87b9/volumes"
Sep 07 04:11:52.815422 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:52.815382    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:52.916176 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:52.916144    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:53.095515 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:53.095376    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:11:53.524024 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:11:53.523993    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:03.790873 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:03.790839    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"]
Sep 07 04:12:03.830300 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:03.830264    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:03.947159 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:03.947124    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:03.955505 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:03.955475    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t"]
Sep 07 04:12:03.955730 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:03.955701    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t" podUID="6103b3e4-bc89-4ccb-858b-0ba09f16aba1" containerName="payload-processing" containerID="cri-o://57ffd88f351506317126a4aec6d5399e5689dc7eeb80081f25690645a42f0833" gracePeriod=30
Sep 07 04:12:03.962403 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:03.962376    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9"]
Sep 07 04:12:03.963079 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:03.962629    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9" podUID="f4d9c33f-f46c-40e9-887b-b4a303669bdb" containerName="payload-pre-processing" containerID="cri-o://45a6cffb74e4e86a19d70c0b95ba95c9a16a0b309b65835a0b44640a75a0d903" gracePeriod=30
Sep 07 04:12:04.047074 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.046997    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:04.107178 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.107149    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:04.220401 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.220377    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t"
Sep 07 04:12:04.236024 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.235987    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9"
Sep 07 04:12:04.270832 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.270802    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4d9c33f-f46c-40e9-887b-b4a303669bdb-plugins-config-volume\") pod \"f4d9c33f-f46c-40e9-887b-b4a303669bdb\" (UID: \"f4d9c33f-f46c-40e9-887b-b4a303669bdb\") "
Sep 07 04:12:04.270832 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.270837    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjgsf\" (UniqueName: \"kubernetes.io/projected/f4d9c33f-f46c-40e9-887b-b4a303669bdb-kube-api-access-bjgsf\") pod \"f4d9c33f-f46c-40e9-887b-b4a303669bdb\" (UID: \"f4d9c33f-f46c-40e9-887b-b4a303669bdb\") "
Sep 07 04:12:04.271098 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.270882    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6103b3e4-bc89-4ccb-858b-0ba09f16aba1-plugins-config-volume\") pod \"6103b3e4-bc89-4ccb-858b-0ba09f16aba1\" (UID: \"6103b3e4-bc89-4ccb-858b-0ba09f16aba1\") "
Sep 07 04:12:04.271098 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.270937    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8dbf\" (UniqueName: \"kubernetes.io/projected/6103b3e4-bc89-4ccb-858b-0ba09f16aba1-kube-api-access-w8dbf\") pod \"6103b3e4-bc89-4ccb-858b-0ba09f16aba1\" (UID: \"6103b3e4-bc89-4ccb-858b-0ba09f16aba1\") "
Sep 07 04:12:04.271288 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.271265    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4d9c33f-f46c-40e9-887b-b4a303669bdb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f4d9c33f-f46c-40e9-887b-b4a303669bdb" (UID: "f4d9c33f-f46c-40e9-887b-b4a303669bdb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 04:12:04.271351 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.271319    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6103b3e4-bc89-4ccb-858b-0ba09f16aba1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6103b3e4-bc89-4ccb-858b-0ba09f16aba1" (UID: "6103b3e4-bc89-4ccb-858b-0ba09f16aba1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 04:12:04.273226 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.273190    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4d9c33f-f46c-40e9-887b-b4a303669bdb-kube-api-access-bjgsf" (OuterVolumeSpecName: "kube-api-access-bjgsf") pod "f4d9c33f-f46c-40e9-887b-b4a303669bdb" (UID: "f4d9c33f-f46c-40e9-887b-b4a303669bdb"). InnerVolumeSpecName "kube-api-access-bjgsf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:12:04.273319 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.273203    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6103b3e4-bc89-4ccb-858b-0ba09f16aba1-kube-api-access-w8dbf" (OuterVolumeSpecName: "kube-api-access-w8dbf") pod "6103b3e4-bc89-4ccb-858b-0ba09f16aba1" (UID: "6103b3e4-bc89-4ccb-858b-0ba09f16aba1"). InnerVolumeSpecName "kube-api-access-w8dbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:12:04.371649 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.371568    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4d9c33f-f46c-40e9-887b-b4a303669bdb-plugins-config-volume\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:12:04.371649 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.371598    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjgsf\" (UniqueName: \"kubernetes.io/projected/f4d9c33f-f46c-40e9-887b-b4a303669bdb-kube-api-access-bjgsf\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:12:04.371649 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.371608    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6103b3e4-bc89-4ccb-858b-0ba09f16aba1-plugins-config-volume\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:12:04.371649 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.371618    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8dbf\" (UniqueName: \"kubernetes.io/projected/6103b3e4-bc89-4ccb-858b-0ba09f16aba1-kube-api-access-w8dbf\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:12:04.482831 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.482794    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:04.597976 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.597937    2583 generic.go:356] "Generic (PLEG): container finished" podID="6103b3e4-bc89-4ccb-858b-0ba09f16aba1" containerID="57ffd88f351506317126a4aec6d5399e5689dc7eeb80081f25690645a42f0833" exitCode=0
Sep 07 04:12:04.598168 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.597981    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t"
Sep 07 04:12:04.598168 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.598032    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t" event={"ID":"6103b3e4-bc89-4ccb-858b-0ba09f16aba1","Type":"ContainerDied","Data":"57ffd88f351506317126a4aec6d5399e5689dc7eeb80081f25690645a42f0833"}
Sep 07 04:12:04.598168 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.598081    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t" event={"ID":"6103b3e4-bc89-4ccb-858b-0ba09f16aba1","Type":"ContainerDied","Data":"d5b18b29982a4bb69d03a4d6e793fffe48c34ed5abf7b2bbfa477d60f48a7e4a"}
Sep 07 04:12:04.598168 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.598106    2583 scope.go:117] "RemoveContainer" containerID="57ffd88f351506317126a4aec6d5399e5689dc7eeb80081f25690645a42f0833"
Sep 07 04:12:04.599491 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.599469    2583 generic.go:356] "Generic (PLEG): container finished" podID="f4d9c33f-f46c-40e9-887b-b4a303669bdb" containerID="45a6cffb74e4e86a19d70c0b95ba95c9a16a0b309b65835a0b44640a75a0d903" exitCode=0
Sep 07 04:12:04.599604 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.599512    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9"
Sep 07 04:12:04.599604 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.599552    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9" event={"ID":"f4d9c33f-f46c-40e9-887b-b4a303669bdb","Type":"ContainerDied","Data":"45a6cffb74e4e86a19d70c0b95ba95c9a16a0b309b65835a0b44640a75a0d903"}
Sep 07 04:12:04.599703 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.599617    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9" event={"ID":"f4d9c33f-f46c-40e9-887b-b4a303669bdb","Type":"ContainerDied","Data":"89a133c69e6bb0f9b683e365239aac0ee3e8a95cb4d7e96af234605893f00c88"}
Sep 07 04:12:04.608496 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.608477    2583 scope.go:117] "RemoveContainer" containerID="57ffd88f351506317126a4aec6d5399e5689dc7eeb80081f25690645a42f0833"
Sep 07 04:12:04.608844 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:12:04.608823    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57ffd88f351506317126a4aec6d5399e5689dc7eeb80081f25690645a42f0833\": container with ID starting with 57ffd88f351506317126a4aec6d5399e5689dc7eeb80081f25690645a42f0833 not found: ID does not exist" containerID="57ffd88f351506317126a4aec6d5399e5689dc7eeb80081f25690645a42f0833"
Sep 07 04:12:04.608938 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.608852    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57ffd88f351506317126a4aec6d5399e5689dc7eeb80081f25690645a42f0833"} err="failed to get container status \"57ffd88f351506317126a4aec6d5399e5689dc7eeb80081f25690645a42f0833\": rpc error: code = NotFound desc = could not find container \"57ffd88f351506317126a4aec6d5399e5689dc7eeb80081f25690645a42f0833\": container with ID starting with 57ffd88f351506317126a4aec6d5399e5689dc7eeb80081f25690645a42f0833 not found: ID does not exist"
Sep 07 04:12:04.608938 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.608870    2583 scope.go:117] "RemoveContainer" containerID="45a6cffb74e4e86a19d70c0b95ba95c9a16a0b309b65835a0b44640a75a0d903"
Sep 07 04:12:04.617315 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.617298    2583 scope.go:117] "RemoveContainer" containerID="45a6cffb74e4e86a19d70c0b95ba95c9a16a0b309b65835a0b44640a75a0d903"
Sep 07 04:12:04.617590 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:12:04.617571    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45a6cffb74e4e86a19d70c0b95ba95c9a16a0b309b65835a0b44640a75a0d903\": container with ID starting with 45a6cffb74e4e86a19d70c0b95ba95c9a16a0b309b65835a0b44640a75a0d903 not found: ID does not exist" containerID="45a6cffb74e4e86a19d70c0b95ba95c9a16a0b309b65835a0b44640a75a0d903"
Sep 07 04:12:04.617631 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.617598    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45a6cffb74e4e86a19d70c0b95ba95c9a16a0b309b65835a0b44640a75a0d903"} err="failed to get container status \"45a6cffb74e4e86a19d70c0b95ba95c9a16a0b309b65835a0b44640a75a0d903\": rpc error: code = NotFound desc = could not find container \"45a6cffb74e4e86a19d70c0b95ba95c9a16a0b309b65835a0b44640a75a0d903\": container with ID starting with 45a6cffb74e4e86a19d70c0b95ba95c9a16a0b309b65835a0b44640a75a0d903 not found: ID does not exist"
Sep 07 04:12:04.624694 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.624631    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t"]
Sep 07 04:12:04.629569 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.629536    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-a89024-5d974c4567-7sx4t"]
Sep 07 04:12:04.638730 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.638702    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9"]
Sep 07 04:12:04.642230 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:04.642190    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a89024-c9596ddbb-fhnd9"]
Sep 07 04:12:05.522182 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:05.522150    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6103b3e4-bc89-4ccb-858b-0ba09f16aba1" path="/var/lib/kubelet/pods/6103b3e4-bc89-4ccb-858b-0ba09f16aba1/volumes"
Sep 07 04:12:05.522567 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:05.522519    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4d9c33f-f46c-40e9-887b-b4a303669bdb" path="/var/lib/kubelet/pods/f4d9c33f-f46c-40e9-887b-b4a303669bdb/volumes"
Sep 07 04:12:06.092125 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:06.092088    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:06.563632 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:06.563599    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:06.877274 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:06.877170    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:07.072611 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:07.072569    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:09.572422 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:09.572384    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:09.670360 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:09.670322    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:10.710890 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.710856    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:10.773954 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.773921    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:10.778342 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.778315    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"]
Sep 07 04:12:10.778855 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.778835    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b57da335-bd2e-4350-a62d-3e1598b45feb" containerName="maas-api"
Sep 07 04:12:10.778995 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.778911    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b57da335-bd2e-4350-a62d-3e1598b45feb" containerName="maas-api"
Sep 07 04:12:10.778995 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.778931    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fdee0a9-1427-4f4a-897d-594ea81e87b9" containerName="maas-api"
Sep 07 04:12:10.778995 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.778939    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fdee0a9-1427-4f4a-897d-594ea81e87b9" containerName="maas-api"
Sep 07 04:12:10.778995 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.778950    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6103b3e4-bc89-4ccb-858b-0ba09f16aba1" containerName="payload-processing"
Sep 07 04:12:10.778995 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.778958    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6103b3e4-bc89-4ccb-858b-0ba09f16aba1" containerName="payload-processing"
Sep 07 04:12:10.778995 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.778995    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4d9c33f-f46c-40e9-887b-b4a303669bdb" containerName="payload-pre-processing"
Sep 07 04:12:10.779336 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.779003    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4d9c33f-f46c-40e9-887b-b4a303669bdb" containerName="payload-pre-processing"
Sep 07 04:12:10.779336 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.779015    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdf8eea2-7026-4e38-acbb-c0aa3efdbe26" containerName="payload-pre-processing"
Sep 07 04:12:10.779336 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.779023    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdf8eea2-7026-4e38-acbb-c0aa3efdbe26" containerName="payload-pre-processing"
Sep 07 04:12:10.779336 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.779040    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daa7327d-1d16-4d4a-b8ca-bf1e86d0af98" containerName="payload-pre-processing"
Sep 07 04:12:10.779336 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.779049    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="daa7327d-1d16-4d4a-b8ca-bf1e86d0af98" containerName="payload-pre-processing"
Sep 07 04:12:10.779336 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.779149    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b57da335-bd2e-4350-a62d-3e1598b45feb" containerName="maas-api"
Sep 07 04:12:10.779336 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.779163    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="daa7327d-1d16-4d4a-b8ca-bf1e86d0af98" containerName="payload-pre-processing"
Sep 07 04:12:10.779336 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.779174    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4d9c33f-f46c-40e9-887b-b4a303669bdb" containerName="payload-pre-processing"
Sep 07 04:12:10.779336 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.779186    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fdee0a9-1427-4f4a-897d-594ea81e87b9" containerName="maas-api"
Sep 07 04:12:10.779336 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.779196    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6103b3e4-bc89-4ccb-858b-0ba09f16aba1" containerName="payload-processing"
Sep 07 04:12:10.779336 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.779206    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdf8eea2-7026-4e38-acbb-c0aa3efdbe26" containerName="payload-pre-processing"
Sep 07 04:12:10.784367 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.784341    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"
Sep 07 04:12:10.786556 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.786496    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-f8fdfa-serving-cert\""
Sep 07 04:12:10.793204 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.793178    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"]
Sep 07 04:12:10.826085 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.826051    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83cc94fd-1c50-4132-aa42-a2cb01445656-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5\" (UID: \"83cc94fd-1c50-4132-aa42-a2cb01445656\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"
Sep 07 04:12:10.826323 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.826281    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83cc94fd-1c50-4132-aa42-a2cb01445656-metrics-certs\") pod \"maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5\" (UID: \"83cc94fd-1c50-4132-aa42-a2cb01445656\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"
Sep 07 04:12:10.826323 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.826315    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7c2rr\" (UniqueName: \"kubernetes.io/projected/83cc94fd-1c50-4132-aa42-a2cb01445656-kube-api-access-7c2rr\") pod \"maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5\" (UID: \"83cc94fd-1c50-4132-aa42-a2cb01445656\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"
Sep 07 04:12:10.826448 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.826349    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83cc94fd-1c50-4132-aa42-a2cb01445656-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5\" (UID: \"83cc94fd-1c50-4132-aa42-a2cb01445656\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"
Sep 07 04:12:10.837579 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.837551    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:10.927710 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.927675    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83cc94fd-1c50-4132-aa42-a2cb01445656-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5\" (UID: \"83cc94fd-1c50-4132-aa42-a2cb01445656\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"
Sep 07 04:12:10.927926 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.927743    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83cc94fd-1c50-4132-aa42-a2cb01445656-metrics-certs\") pod \"maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5\" (UID: \"83cc94fd-1c50-4132-aa42-a2cb01445656\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"
Sep 07 04:12:10.927926 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.927762    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c2rr\" (UniqueName: \"kubernetes.io/projected/83cc94fd-1c50-4132-aa42-a2cb01445656-kube-api-access-7c2rr\") pod \"maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5\" (UID: \"83cc94fd-1c50-4132-aa42-a2cb01445656\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"
Sep 07 04:12:10.927926 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.927784    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83cc94fd-1c50-4132-aa42-a2cb01445656-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5\" (UID: \"83cc94fd-1c50-4132-aa42-a2cb01445656\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"
Sep 07 04:12:10.927926 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:12:10.927836    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-serving-cert: secret "maas-api-e2e-ipp-a-f8fdfa-serving-cert" not found
Sep 07 04:12:10.928136 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:12:10.927930    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83cc94fd-1c50-4132-aa42-a2cb01445656-maas-api-tls podName:83cc94fd-1c50-4132-aa42-a2cb01445656 nodeName:}" failed. No retries permitted until 2026-09-07 04:12:11.4279088 +0000 UTC m=+1418.361389500 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/83cc94fd-1c50-4132-aa42-a2cb01445656-maas-api-tls") pod "maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5" (UID: "83cc94fd-1c50-4132-aa42-a2cb01445656") : secret "maas-api-e2e-ipp-a-f8fdfa-serving-cert" not found
Sep 07 04:12:10.928420 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.928403    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83cc94fd-1c50-4132-aa42-a2cb01445656-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5\" (UID: \"83cc94fd-1c50-4132-aa42-a2cb01445656\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"
Sep 07 04:12:10.930489 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.930459    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83cc94fd-1c50-4132-aa42-a2cb01445656-metrics-certs\") pod \"maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5\" (UID: \"83cc94fd-1c50-4132-aa42-a2cb01445656\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"
Sep 07 04:12:10.935769 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:10.935742    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7c2rr\" (UniqueName: \"kubernetes.io/projected/83cc94fd-1c50-4132-aa42-a2cb01445656-kube-api-access-7c2rr\") pod \"maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5\" (UID: \"83cc94fd-1c50-4132-aa42-a2cb01445656\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"
Sep 07 04:12:11.012823 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:11.012735    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:11.222202 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:11.222161    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:11.346082 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:11.346001    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:11.433769 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:11.433737    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83cc94fd-1c50-4132-aa42-a2cb01445656-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5\" (UID: \"83cc94fd-1c50-4132-aa42-a2cb01445656\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"
Sep 07 04:12:11.436537 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:11.436509    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83cc94fd-1c50-4132-aa42-a2cb01445656-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5\" (UID: \"83cc94fd-1c50-4132-aa42-a2cb01445656\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"
Sep 07 04:12:11.441085 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:11.441060    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:11.696584 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:11.696552    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"
Sep 07 04:12:11.970703 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:11.970673    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"]
Sep 07 04:12:11.973065 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:12:11.973040    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83cc94fd_1c50_4132_aa42_a2cb01445656.slice/crio-1aa32e5616a632a8655b11755e36f915a47b63d8c46372a8f153a7a8212ae200 WatchSource:0}: Error finding container 1aa32e5616a632a8655b11755e36f915a47b63d8c46372a8f153a7a8212ae200: Status 404 returned error can't find the container with id 1aa32e5616a632a8655b11755e36f915a47b63d8c46372a8f153a7a8212ae200
Sep 07 04:12:11.974433 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:11.974413    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 04:12:12.648034 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:12.648000    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5" event={"ID":"83cc94fd-1c50-4132-aa42-a2cb01445656","Type":"ContainerStarted","Data":"210b98c361776950f5052bc2226a33c9e44e81f61d71efdafdc431a097a4b67e"}
Sep 07 04:12:12.648283 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:12.648095    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5" event={"ID":"83cc94fd-1c50-4132-aa42-a2cb01445656","Type":"ContainerStarted","Data":"1aa32e5616a632a8655b11755e36f915a47b63d8c46372a8f153a7a8212ae200"}
Sep 07 04:12:12.648283 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:12.648124    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"
Sep 07 04:12:12.664890 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:12.664829    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5" podStartSLOduration=2.202083904 podStartE2EDuration="2.664809354s" totalImagesPullingTime="462.72545ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:12:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:12:12.662962183 +0000 UTC m=+1419.596442901" watchObservedRunningTime="2026-09-07 04:12:12.664809354 +0000 UTC m=+1419.598290072"
Sep 07 04:12:18.103821 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:18.103787    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:18.658845 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:18.658819    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"
Sep 07 04:12:18.811170 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:18.811134    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2" podUID="697c4957-ab95-41c1-9f44-fa5c125cb7a0" containerName="main" containerID="cri-o://64fca9beccbe5629469aa695228b2eafe8c9fcd096ac8dc8bf47a45928e2fe06" gracePeriod=45
Sep 07 04:12:19.067654 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.067629    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:12:19.105765 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.105733    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-tmp-dir\") pod \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") "
Sep 07 04:12:19.106272 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.105778    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-dshm\") pod \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") "
Sep 07 04:12:19.106272 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.105802    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krgwc\" (UniqueName: \"kubernetes.io/projected/697c4957-ab95-41c1-9f44-fa5c125cb7a0-kube-api-access-krgwc\") pod \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") "
Sep 07 04:12:19.106272 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.105829    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-model-cache\") pod \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") "
Sep 07 04:12:19.106272 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.105872    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/697c4957-ab95-41c1-9f44-fa5c125cb7a0-tls-certs\") pod \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") "
Sep 07 04:12:19.106272 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.105953    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-home\") pod \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\" (UID: \"697c4957-ab95-41c1-9f44-fa5c125cb7a0\") "
Sep 07 04:12:19.106272 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.106083    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "697c4957-ab95-41c1-9f44-fa5c125cb7a0" (UID: "697c4957-ab95-41c1-9f44-fa5c125cb7a0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:12:19.106272 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.106124    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-model-cache" (OuterVolumeSpecName: "model-cache") pod "697c4957-ab95-41c1-9f44-fa5c125cb7a0" (UID: "697c4957-ab95-41c1-9f44-fa5c125cb7a0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:12:19.106803 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.106296    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-home" (OuterVolumeSpecName: "home") pod "697c4957-ab95-41c1-9f44-fa5c125cb7a0" (UID: "697c4957-ab95-41c1-9f44-fa5c125cb7a0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:12:19.106803 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.106320    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-model-cache\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:12:19.106803 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.106341    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-tmp-dir\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:12:19.108241 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.108181    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-dshm" (OuterVolumeSpecName: "dshm") pod "697c4957-ab95-41c1-9f44-fa5c125cb7a0" (UID: "697c4957-ab95-41c1-9f44-fa5c125cb7a0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:12:19.108361 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.108247    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/697c4957-ab95-41c1-9f44-fa5c125cb7a0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "697c4957-ab95-41c1-9f44-fa5c125cb7a0" (UID: "697c4957-ab95-41c1-9f44-fa5c125cb7a0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:12:19.108361 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.108281    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/697c4957-ab95-41c1-9f44-fa5c125cb7a0-kube-api-access-krgwc" (OuterVolumeSpecName: "kube-api-access-krgwc") pod "697c4957-ab95-41c1-9f44-fa5c125cb7a0" (UID: "697c4957-ab95-41c1-9f44-fa5c125cb7a0"). InnerVolumeSpecName "kube-api-access-krgwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:12:19.207162 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.207132    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-home\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:12:19.207162 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.207159    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/697c4957-ab95-41c1-9f44-fa5c125cb7a0-dshm\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:12:19.207388 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.207171    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krgwc\" (UniqueName: \"kubernetes.io/projected/697c4957-ab95-41c1-9f44-fa5c125cb7a0-kube-api-access-krgwc\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:12:19.207388 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.207184    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/697c4957-ab95-41c1-9f44-fa5c125cb7a0-tls-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:12:19.686064 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.686027    2583 generic.go:356] "Generic (PLEG): container finished" podID="697c4957-ab95-41c1-9f44-fa5c125cb7a0" containerID="64fca9beccbe5629469aa695228b2eafe8c9fcd096ac8dc8bf47a45928e2fe06" exitCode=0
Sep 07 04:12:19.686253 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.686079    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"
Sep 07 04:12:19.686253 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.686119    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2" event={"ID":"697c4957-ab95-41c1-9f44-fa5c125cb7a0","Type":"ContainerDied","Data":"64fca9beccbe5629469aa695228b2eafe8c9fcd096ac8dc8bf47a45928e2fe06"}
Sep 07 04:12:19.686253 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.686161    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2" event={"ID":"697c4957-ab95-41c1-9f44-fa5c125cb7a0","Type":"ContainerDied","Data":"2bb10b6ec575c05a74706286b187811feeec7653e63f47cea7b1b3dabad316e7"}
Sep 07 04:12:19.686253 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.686179    2583 scope.go:117] "RemoveContainer" containerID="64fca9beccbe5629469aa695228b2eafe8c9fcd096ac8dc8bf47a45928e2fe06"
Sep 07 04:12:19.694873 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.694850    2583 scope.go:117] "RemoveContainer" containerID="64fca9beccbe5629469aa695228b2eafe8c9fcd096ac8dc8bf47a45928e2fe06"
Sep 07 04:12:19.695180 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:12:19.695158    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64fca9beccbe5629469aa695228b2eafe8c9fcd096ac8dc8bf47a45928e2fe06\": container with ID starting with 64fca9beccbe5629469aa695228b2eafe8c9fcd096ac8dc8bf47a45928e2fe06 not found: ID does not exist" containerID="64fca9beccbe5629469aa695228b2eafe8c9fcd096ac8dc8bf47a45928e2fe06"
Sep 07 04:12:19.695348 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.695190    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64fca9beccbe5629469aa695228b2eafe8c9fcd096ac8dc8bf47a45928e2fe06"} err="failed to get container status \"64fca9beccbe5629469aa695228b2eafe8c9fcd096ac8dc8bf47a45928e2fe06\": rpc error: code = NotFound desc = could not find container \"64fca9beccbe5629469aa695228b2eafe8c9fcd096ac8dc8bf47a45928e2fe06\": container with ID starting with 64fca9beccbe5629469aa695228b2eafe8c9fcd096ac8dc8bf47a45928e2fe06 not found: ID does not exist"
Sep 07 04:12:19.702365 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.702338    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"]
Sep 07 04:12:19.706710 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:19.706687    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b8cea6/rate-test-model-b8cea6-kserve-f9b6d696c-89jh2"]
Sep 07 04:12:21.521879 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:21.521847    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="697c4957-ab95-41c1-9f44-fa5c125cb7a0" path="/var/lib/kubelet/pods/697c4957-ab95-41c1-9f44-fa5c125cb7a0/volumes"
Sep 07 04:12:32.386553 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:32.386466    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:34.684390 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:34.684350    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:35.448496 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:35.448461    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:35.775981 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:35.775894    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:36.091946 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:36.091864    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:38.753899 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:38.753863    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:38.854086 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:38.854053    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:39.898903 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:39.898860    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:39.956749 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:39.956723    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:40.059898 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:40.059864    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:40.371945 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:40.371913    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:40.463784 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:40.463756    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:12:40.464822 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:40.464798    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj"]
Sep 07 04:12:40.465269 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:40.465256    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="697c4957-ab95-41c1-9f44-fa5c125cb7a0" containerName="main"
Sep 07 04:12:40.465327 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:40.465272    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="697c4957-ab95-41c1-9f44-fa5c125cb7a0" containerName="main"
Sep 07 04:12:40.465363 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:40.465356    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="697c4957-ab95-41c1-9f44-fa5c125cb7a0" containerName="main"
Sep 07 04:12:40.468449 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:40.468433    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj"
Sep 07 04:12:40.470900 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:40.470872    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-704b16\""
Sep 07 04:12:40.471014 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:40.470993    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-704b16-dockercfg-j8n6f\""
Sep 07 04:12:40.478366 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:40.478343    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj"]
Sep 07 04:12:40.616223 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:40.616176    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3003c51a-85fe-47d9-87b5-4639378d85ce-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj\" (UID: \"3003c51a-85fe-47d9-87b5-4639378d85ce\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj"
Sep 07 04:12:40.616406 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:40.616251    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmdbv\" (UniqueName: \"kubernetes.io/projected/3003c51a-85fe-47d9-87b5-4639378d85ce-kube-api-access-mmdbv\") pod \"payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj\" (UID: \"3003c51a-85fe-47d9-87b5-4639378d85ce\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj"
Sep 07 04:12:40.717442 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:40.717404    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3003c51a-85fe-47d9-87b5-4639378d85ce-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj\" (UID: \"3003c51a-85fe-47d9-87b5-4639378d85ce\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj"
Sep 07 04:12:40.717655 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:40.717458    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmdbv\" (UniqueName: \"kubernetes.io/projected/3003c51a-85fe-47d9-87b5-4639378d85ce-kube-api-access-mmdbv\") pod \"payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj\" (UID: \"3003c51a-85fe-47d9-87b5-4639378d85ce\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj"
Sep 07 04:12:40.718181 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:40.718158    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3003c51a-85fe-47d9-87b5-4639378d85ce-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj\" (UID: \"3003c51a-85fe-47d9-87b5-4639378d85ce\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj"
Sep 07 04:12:40.724900 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:40.724866    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmdbv\" (UniqueName: \"kubernetes.io/projected/3003c51a-85fe-47d9-87b5-4639378d85ce-kube-api-access-mmdbv\") pod \"payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj\" (UID: \"3003c51a-85fe-47d9-87b5-4639378d85ce\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj"
Sep 07 04:12:40.779976 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:40.779943    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj"
Sep 07 04:12:41.050160 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:41.050135    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj"]
Sep 07 04:12:41.052286 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:12:41.052261    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3003c51a_85fe_47d9_87b5_4639378d85ce.slice/crio-22aa76a25a3b848b120a7a4c724e5f052d898ae6fd1e228cbe41dc921fa8ae2e WatchSource:0}: Error finding container 22aa76a25a3b848b120a7a4c724e5f052d898ae6fd1e228cbe41dc921fa8ae2e: Status 404 returned error can't find the container with id 22aa76a25a3b848b120a7a4c724e5f052d898ae6fd1e228cbe41dc921fa8ae2e
Sep 07 04:12:41.804142 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:41.804108    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj" event={"ID":"3003c51a-85fe-47d9-87b5-4639378d85ce","Type":"ContainerStarted","Data":"744a8ef9648ad3cc156b3183dfc31662a7933c01ffc27ea118c2ae5297e59046"}
Sep 07 04:12:41.804357 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:41.804157    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj" event={"ID":"3003c51a-85fe-47d9-87b5-4639378d85ce","Type":"ContainerStarted","Data":"22aa76a25a3b848b120a7a4c724e5f052d898ae6fd1e228cbe41dc921fa8ae2e"}
Sep 07 04:12:41.804357 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:41.804250    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj"
Sep 07 04:12:41.823854 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:41.823802    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj" podStartSLOduration=1.82378465 podStartE2EDuration="1.82378465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:12:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:12:41.821949127 +0000 UTC m=+1448.755429845" watchObservedRunningTime="2026-09-07 04:12:41.82378465 +0000 UTC m=+1448.757265390"
Sep 07 04:12:52.812398 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:12:52.812369    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj"
Sep 07 04:13:03.785636 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:03.785603    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:13:11.662717 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.662658    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"]
Sep 07 04:13:11.666505 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.666486    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:11.668832 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.668803    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-f8fdfa\"/\"kube-root-ca.crt\""
Sep 07 04:13:11.669770 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.669753    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-f8fdfa\"/\"openshift-service-ca.crt\""
Sep 07 04:13:11.669932 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.669915    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-f8fdfa\"/\"ipp-route-f8fdfa-kserve-self-signed-certs\""
Sep 07 04:13:11.670013 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.670001    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-f8fdfa\"/\"default-dockercfg-n8dpx\""
Sep 07 04:13:11.675871 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.675633    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"]
Sep 07 04:13:11.682386 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.682359    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:13:11.811523 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.811489    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-model-cache\") pod \"ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") " pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:11.811724 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.811541    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4jp7\" (UniqueName: \"kubernetes.io/projected/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-kube-api-access-k4jp7\") pod \"ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") " pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:11.811724 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.811617    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-tls-certs\") pod \"ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") " pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:11.811829 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.811723    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-home\") pod \"ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") " pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:11.811829 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.811750    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-dshm\") pod \"ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") " pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:11.811829 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.811781    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-tmp-dir\") pod \"ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") " pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:11.913110 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.913026    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-model-cache\") pod \"ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") " pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:11.913110 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.913074    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4jp7\" (UniqueName: \"kubernetes.io/projected/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-kube-api-access-k4jp7\") pod \"ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") " pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:11.913110 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.913109    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-tls-certs\") pod \"ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") " pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:11.913494 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.913190    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-home\") pod \"ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") " pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:11.913494 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.913255    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-dshm\") pod \"ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") " pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:11.913494 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.913299    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-tmp-dir\") pod \"ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") " pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:11.913659 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.913531    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-model-cache\") pod \"ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") " pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:11.913659 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.913597    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-tmp-dir\") pod \"ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") " pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:11.913659 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.913605    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-home\") pod \"ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") " pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:11.915578 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.915553    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-dshm\") pod \"ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") " pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:11.915823 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.915808    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-tls-certs\") pod \"ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") " pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:11.921969 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.921938    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4jp7\" (UniqueName: \"kubernetes.io/projected/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-kube-api-access-k4jp7\") pod \"ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") " pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:11.978279 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:11.978249    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:12.250779 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:12.250750    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"]
Sep 07 04:13:12.253458 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:13:12.253427    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5c5ce30_34a0_4b11_a13c_0bdb60e98819.slice/crio-311c9db29e8bdceb1b848ad05fbe9584caa741b62f3132c0d67e38ef0f7f7db9 WatchSource:0}: Error finding container 311c9db29e8bdceb1b848ad05fbe9584caa741b62f3132c0d67e38ef0f7f7db9: Status 404 returned error can't find the container with id 311c9db29e8bdceb1b848ad05fbe9584caa741b62f3132c0d67e38ef0f7f7db9
Sep 07 04:13:12.961991 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:12.961954    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw" event={"ID":"b5c5ce30-34a0-4b11-a13c-0bdb60e98819","Type":"ContainerStarted","Data":"40ccc4fbc15b560c3b1e94eab257221277a8f51bc53c1715ba3053691954155d"}
Sep 07 04:13:12.962413 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:12.962003    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw" event={"ID":"b5c5ce30-34a0-4b11-a13c-0bdb60e98819","Type":"ContainerStarted","Data":"311c9db29e8bdceb1b848ad05fbe9584caa741b62f3132c0d67e38ef0f7f7db9"}
Sep 07 04:13:12.978598 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:12.978548    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw" podStartSLOduration=1.9785297179999999 podStartE2EDuration="1.978529718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:13:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:13:12.977150223 +0000 UTC m=+1479.910630940" watchObservedRunningTime="2026-09-07 04:13:12.978529718 +0000 UTC m=+1479.912010435"
Sep 07 04:13:21.107406 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:21.107329    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:13:21.978837 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:21.978791    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:21.979050 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:21.978888    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:21.991421 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:21.991390    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:22.018245 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:22.018184    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:13:34.766488 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:34.766448    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:13:35.064634 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:35.064558    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:13:35.228616 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:35.228585    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:13:38.027280 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:38.027241    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:13:38.118801 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:38.118768    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:13:39.117862 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.117825    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"]
Sep 07 04:13:39.121937 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.121909    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"
Sep 07 04:13:39.124367 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.124333    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-c5ce6a-serving-cert\""
Sep 07 04:13:39.133006 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.132937    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"]
Sep 07 04:13:39.153438 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.153383    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:13:39.180352 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.180325    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9b38651-3b91-4da0-9e73-c432bf63f2ec-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf\" (UID: \"a9b38651-3b91-4da0-9e73-c432bf63f2ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"
Sep 07 04:13:39.180549 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.180376    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9b38651-3b91-4da0-9e73-c432bf63f2ec-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf\" (UID: \"a9b38651-3b91-4da0-9e73-c432bf63f2ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"
Sep 07 04:13:39.180549 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.180463    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b38651-3b91-4da0-9e73-c432bf63f2ec-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf\" (UID: \"a9b38651-3b91-4da0-9e73-c432bf63f2ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"
Sep 07 04:13:39.180664 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.180547    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhgtd\" (UniqueName: \"kubernetes.io/projected/a9b38651-3b91-4da0-9e73-c432bf63f2ec-kube-api-access-jhgtd\") pod \"maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf\" (UID: \"a9b38651-3b91-4da0-9e73-c432bf63f2ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"
Sep 07 04:13:39.252671 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.252640    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:13:39.281174 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.281139    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhgtd\" (UniqueName: \"kubernetes.io/projected/a9b38651-3b91-4da0-9e73-c432bf63f2ec-kube-api-access-jhgtd\") pod \"maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf\" (UID: \"a9b38651-3b91-4da0-9e73-c432bf63f2ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"
Sep 07 04:13:39.281376 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.281195    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9b38651-3b91-4da0-9e73-c432bf63f2ec-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf\" (UID: \"a9b38651-3b91-4da0-9e73-c432bf63f2ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"
Sep 07 04:13:39.281376 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.281286    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9b38651-3b91-4da0-9e73-c432bf63f2ec-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf\" (UID: \"a9b38651-3b91-4da0-9e73-c432bf63f2ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"
Sep 07 04:13:39.281376 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.281350    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b38651-3b91-4da0-9e73-c432bf63f2ec-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf\" (UID: \"a9b38651-3b91-4da0-9e73-c432bf63f2ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"
Sep 07 04:13:39.281539 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:13:39.281421    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-serving-cert: secret "maas-api-e2e-ipp-cleanup-c5ce6a-serving-cert" not found
Sep 07 04:13:39.281539 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:13:39.281508    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9b38651-3b91-4da0-9e73-c432bf63f2ec-maas-api-tls podName:a9b38651-3b91-4da0-9e73-c432bf63f2ec nodeName:}" failed. No retries permitted until 2026-09-07 04:13:39.781486794 +0000 UTC m=+1506.714967490 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a9b38651-3b91-4da0-9e73-c432bf63f2ec-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf" (UID: "a9b38651-3b91-4da0-9e73-c432bf63f2ec") : secret "maas-api-e2e-ipp-cleanup-c5ce6a-serving-cert" not found
Sep 07 04:13:39.281968 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.281947    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b38651-3b91-4da0-9e73-c432bf63f2ec-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf\" (UID: \"a9b38651-3b91-4da0-9e73-c432bf63f2ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"
Sep 07 04:13:39.283714 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.283697    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9b38651-3b91-4da0-9e73-c432bf63f2ec-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf\" (UID: \"a9b38651-3b91-4da0-9e73-c432bf63f2ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"
Sep 07 04:13:39.289568 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.289551    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhgtd\" (UniqueName: \"kubernetes.io/projected/a9b38651-3b91-4da0-9e73-c432bf63f2ec-kube-api-access-jhgtd\") pod \"maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf\" (UID: \"a9b38651-3b91-4da0-9e73-c432bf63f2ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"
Sep 07 04:13:39.532373 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.532337    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:13:39.634104 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.634049    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:13:39.735195 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.735135    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:13:39.763621 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.763587    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94"]
Sep 07 04:13:39.767179 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.767158    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94"
Sep 07 04:13:39.769934 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.769911    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-c5ce6a\""
Sep 07 04:13:39.770068 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.769939    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-c5ce6a-dockercfg-kgkhs\""
Sep 07 04:13:39.776925 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.776903    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94"]
Sep 07 04:13:39.785626 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.785551    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9b38651-3b91-4da0-9e73-c432bf63f2ec-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf\" (UID: \"a9b38651-3b91-4da0-9e73-c432bf63f2ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"
Sep 07 04:13:39.788260 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.788237    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9b38651-3b91-4da0-9e73-c432bf63f2ec-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf\" (UID: \"a9b38651-3b91-4da0-9e73-c432bf63f2ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"
Sep 07 04:13:39.816588 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.816563    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:13:39.886270 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.886229    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d6c8fe9-3a0a-4302-8328-9441009045ac-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94\" (UID: \"7d6c8fe9-3a0a-4302-8328-9441009045ac\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94"
Sep 07 04:13:39.886435 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.886342    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cwtw\" (UniqueName: \"kubernetes.io/projected/7d6c8fe9-3a0a-4302-8328-9441009045ac-kube-api-access-7cwtw\") pod \"payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94\" (UID: \"7d6c8fe9-3a0a-4302-8328-9441009045ac\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94"
Sep 07 04:13:39.987719 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.987680    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cwtw\" (UniqueName: \"kubernetes.io/projected/7d6c8fe9-3a0a-4302-8328-9441009045ac-kube-api-access-7cwtw\") pod \"payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94\" (UID: \"7d6c8fe9-3a0a-4302-8328-9441009045ac\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94"
Sep 07 04:13:39.987897 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.987770    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d6c8fe9-3a0a-4302-8328-9441009045ac-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94\" (UID: \"7d6c8fe9-3a0a-4302-8328-9441009045ac\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94"
Sep 07 04:13:39.988473 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.988455    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d6c8fe9-3a0a-4302-8328-9441009045ac-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94\" (UID: \"7d6c8fe9-3a0a-4302-8328-9441009045ac\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94"
Sep 07 04:13:39.995646 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:39.995620    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cwtw\" (UniqueName: \"kubernetes.io/projected/7d6c8fe9-3a0a-4302-8328-9441009045ac-kube-api-access-7cwtw\") pod \"payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94\" (UID: \"7d6c8fe9-3a0a-4302-8328-9441009045ac\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94"
Sep 07 04:13:40.034990 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:40.034954    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"
Sep 07 04:13:40.079399 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:40.079365    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94"
Sep 07 04:13:40.310784 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:40.310755    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"]
Sep 07 04:13:40.313061 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:13:40.313035    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9b38651_3b91_4da0_9e73_c432bf63f2ec.slice/crio-2f86965c9d7a5a32e5f8dff35270dcf92f3c2778fe82307fe8166fa3ade1e976 WatchSource:0}: Error finding container 2f86965c9d7a5a32e5f8dff35270dcf92f3c2778fe82307fe8166fa3ade1e976: Status 404 returned error can't find the container with id 2f86965c9d7a5a32e5f8dff35270dcf92f3c2778fe82307fe8166fa3ade1e976
Sep 07 04:13:40.364056 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:40.364034    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94"]
Sep 07 04:13:40.367112 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:13:40.367081    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d6c8fe9_3a0a_4302_8328_9441009045ac.slice/crio-e431a46dae0647c8d33efc43120aac137369e4e8c0c17725e83ea21f58d5c235 WatchSource:0}: Error finding container e431a46dae0647c8d33efc43120aac137369e4e8c0c17725e83ea21f58d5c235: Status 404 returned error can't find the container with id e431a46dae0647c8d33efc43120aac137369e4e8c0c17725e83ea21f58d5c235
Sep 07 04:13:41.104514 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:41.104475    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94" event={"ID":"7d6c8fe9-3a0a-4302-8328-9441009045ac","Type":"ContainerStarted","Data":"1cfcf6de43332bb0ffc4022bcfbce410f60008ef3878c0eeaea72c7552092e66"}
Sep 07 04:13:41.104726 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:41.104542    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94" event={"ID":"7d6c8fe9-3a0a-4302-8328-9441009045ac","Type":"ContainerStarted","Data":"e431a46dae0647c8d33efc43120aac137369e4e8c0c17725e83ea21f58d5c235"}
Sep 07 04:13:41.104726 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:41.104572    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94"
Sep 07 04:13:41.106041 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:41.106016    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf" event={"ID":"a9b38651-3b91-4da0-9e73-c432bf63f2ec","Type":"ContainerStarted","Data":"49033437d7e36ee7a02631399af5fd9ced3e561bb771daebe2a846cd835abac0"}
Sep 07 04:13:41.106158 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:41.106064    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf" event={"ID":"a9b38651-3b91-4da0-9e73-c432bf63f2ec","Type":"ContainerStarted","Data":"2f86965c9d7a5a32e5f8dff35270dcf92f3c2778fe82307fe8166fa3ade1e976"}
Sep 07 04:13:41.106158 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:41.106083    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"
Sep 07 04:13:41.121202 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:41.121154    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94" podStartSLOduration=2.1211392079999998 podStartE2EDuration="2.121139208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:13:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:13:41.119527506 +0000 UTC m=+1508.053008226" watchObservedRunningTime="2026-09-07 04:13:41.121139208 +0000 UTC m=+1508.054619956"
Sep 07 04:13:41.135275 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:41.135229    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf" podStartSLOduration=1.790058142 podStartE2EDuration="2.135196506s" totalImagesPullingTime="345.138364ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:13:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:13:41.13390903 +0000 UTC m=+1508.067389749" watchObservedRunningTime="2026-09-07 04:13:41.135196506 +0000 UTC m=+1508.068677223"
Sep 07 04:13:47.117711 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:47.117685    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"
Sep 07 04:13:52.114235 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:13:52.114180    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94"
Sep 07 04:14:03.056539 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:03.056441    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:14:09.561998 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.561956    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"]
Sep 07 04:14:09.562608 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.562188    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf" podUID="a9b38651-3b91-4da0-9e73-c432bf63f2ec" containerName="maas-api" containerID="cri-o://49033437d7e36ee7a02631399af5fd9ced3e561bb771daebe2a846cd835abac0" gracePeriod=30
Sep 07 04:14:09.576844 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.576810    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:14:09.606313 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.606277    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94"]
Sep 07 04:14:09.606546 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.606521    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94" podUID="7d6c8fe9-3a0a-4302-8328-9441009045ac" containerName="payload-pre-processing" containerID="cri-o://1cfcf6de43332bb0ffc4022bcfbce410f60008ef3878c0eeaea72c7552092e66" gracePeriod=30
Sep 07 04:14:09.676657 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.676590    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:14:09.748375 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.748332    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:14:09.812083 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.811994    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:14:09.856356 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.856333    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"
Sep 07 04:14:09.878408 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.878377    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9b38651-3b91-4da0-9e73-c432bf63f2ec-metrics-certs\") pod \"a9b38651-3b91-4da0-9e73-c432bf63f2ec\" (UID: \"a9b38651-3b91-4da0-9e73-c432bf63f2ec\") "
Sep 07 04:14:09.878565 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.878429    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9b38651-3b91-4da0-9e73-c432bf63f2ec-maas-api-tls\") pod \"a9b38651-3b91-4da0-9e73-c432bf63f2ec\" (UID: \"a9b38651-3b91-4da0-9e73-c432bf63f2ec\") "
Sep 07 04:14:09.878565 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.878529    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhgtd\" (UniqueName: \"kubernetes.io/projected/a9b38651-3b91-4da0-9e73-c432bf63f2ec-kube-api-access-jhgtd\") pod \"a9b38651-3b91-4da0-9e73-c432bf63f2ec\" (UID: \"a9b38651-3b91-4da0-9e73-c432bf63f2ec\") "
Sep 07 04:14:09.878687 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.878579    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b38651-3b91-4da0-9e73-c432bf63f2ec-openshift-service-ca\") pod \"a9b38651-3b91-4da0-9e73-c432bf63f2ec\" (UID: \"a9b38651-3b91-4da0-9e73-c432bf63f2ec\") "
Sep 07 04:14:09.879240 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.879183    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9b38651-3b91-4da0-9e73-c432bf63f2ec-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a9b38651-3b91-4da0-9e73-c432bf63f2ec" (UID: "a9b38651-3b91-4da0-9e73-c432bf63f2ec"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 04:14:09.881025 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.880999    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9b38651-3b91-4da0-9e73-c432bf63f2ec-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a9b38651-3b91-4da0-9e73-c432bf63f2ec" (UID: "a9b38651-3b91-4da0-9e73-c432bf63f2ec"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:14:09.881189 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.881165    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9b38651-3b91-4da0-9e73-c432bf63f2ec-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a9b38651-3b91-4da0-9e73-c432bf63f2ec" (UID: "a9b38651-3b91-4da0-9e73-c432bf63f2ec"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:14:09.881557 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.881533    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9b38651-3b91-4da0-9e73-c432bf63f2ec-kube-api-access-jhgtd" (OuterVolumeSpecName: "kube-api-access-jhgtd") pod "a9b38651-3b91-4da0-9e73-c432bf63f2ec" (UID: "a9b38651-3b91-4da0-9e73-c432bf63f2ec"). InnerVolumeSpecName "kube-api-access-jhgtd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:14:09.885139 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.885116    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94"
Sep 07 04:14:09.979944 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.979910    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cwtw\" (UniqueName: \"kubernetes.io/projected/7d6c8fe9-3a0a-4302-8328-9441009045ac-kube-api-access-7cwtw\") pod \"7d6c8fe9-3a0a-4302-8328-9441009045ac\" (UID: \"7d6c8fe9-3a0a-4302-8328-9441009045ac\") "
Sep 07 04:14:09.980127 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.979998    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d6c8fe9-3a0a-4302-8328-9441009045ac-plugins-config-volume\") pod \"7d6c8fe9-3a0a-4302-8328-9441009045ac\" (UID: \"7d6c8fe9-3a0a-4302-8328-9441009045ac\") "
Sep 07 04:14:09.980176 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.980170    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9b38651-3b91-4da0-9e73-c432bf63f2ec-metrics-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:14:09.980253 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.980180    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9b38651-3b91-4da0-9e73-c432bf63f2ec-maas-api-tls\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:14:09.980253 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.980189    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhgtd\" (UniqueName: \"kubernetes.io/projected/a9b38651-3b91-4da0-9e73-c432bf63f2ec-kube-api-access-jhgtd\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:14:09.980253 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.980197    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b38651-3b91-4da0-9e73-c432bf63f2ec-openshift-service-ca\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:14:09.980412 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.980389    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d6c8fe9-3a0a-4302-8328-9441009045ac-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7d6c8fe9-3a0a-4302-8328-9441009045ac" (UID: "7d6c8fe9-3a0a-4302-8328-9441009045ac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 04:14:09.982153 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:09.982132    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d6c8fe9-3a0a-4302-8328-9441009045ac-kube-api-access-7cwtw" (OuterVolumeSpecName: "kube-api-access-7cwtw") pod "7d6c8fe9-3a0a-4302-8328-9441009045ac" (UID: "7d6c8fe9-3a0a-4302-8328-9441009045ac"). InnerVolumeSpecName "kube-api-access-7cwtw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:14:10.080993 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:10.080900    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cwtw\" (UniqueName: \"kubernetes.io/projected/7d6c8fe9-3a0a-4302-8328-9441009045ac-kube-api-access-7cwtw\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:14:10.080993 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:10.080933    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d6c8fe9-3a0a-4302-8328-9441009045ac-plugins-config-volume\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:14:10.136450 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:10.136403    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:14:10.254621 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:10.254586    2583 generic.go:356] "Generic (PLEG): container finished" podID="7d6c8fe9-3a0a-4302-8328-9441009045ac" containerID="1cfcf6de43332bb0ffc4022bcfbce410f60008ef3878c0eeaea72c7552092e66" exitCode=0
Sep 07 04:14:10.254796 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:10.254635    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94"
Sep 07 04:14:10.254796 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:10.254675    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94" event={"ID":"7d6c8fe9-3a0a-4302-8328-9441009045ac","Type":"ContainerDied","Data":"1cfcf6de43332bb0ffc4022bcfbce410f60008ef3878c0eeaea72c7552092e66"}
Sep 07 04:14:10.254796 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:10.254728    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94" event={"ID":"7d6c8fe9-3a0a-4302-8328-9441009045ac","Type":"ContainerDied","Data":"e431a46dae0647c8d33efc43120aac137369e4e8c0c17725e83ea21f58d5c235"}
Sep 07 04:14:10.254796 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:10.254759    2583 scope.go:117] "RemoveContainer" containerID="1cfcf6de43332bb0ffc4022bcfbce410f60008ef3878c0eeaea72c7552092e66"
Sep 07 04:14:10.256309 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:10.256276    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"
Sep 07 04:14:10.256309 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:10.256292    2583 generic.go:356] "Generic (PLEG): container finished" podID="a9b38651-3b91-4da0-9e73-c432bf63f2ec" containerID="49033437d7e36ee7a02631399af5fd9ced3e561bb771daebe2a846cd835abac0" exitCode=0
Sep 07 04:14:10.256481 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:10.256329    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf" event={"ID":"a9b38651-3b91-4da0-9e73-c432bf63f2ec","Type":"ContainerDied","Data":"49033437d7e36ee7a02631399af5fd9ced3e561bb771daebe2a846cd835abac0"}
Sep 07 04:14:10.256481 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:10.256367    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf" event={"ID":"a9b38651-3b91-4da0-9e73-c432bf63f2ec","Type":"ContainerDied","Data":"2f86965c9d7a5a32e5f8dff35270dcf92f3c2778fe82307fe8166fa3ade1e976"}
Sep 07 04:14:10.264158 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:10.264140    2583 scope.go:117] "RemoveContainer" containerID="1cfcf6de43332bb0ffc4022bcfbce410f60008ef3878c0eeaea72c7552092e66"
Sep 07 04:14:10.264530 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:14:10.264508    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cfcf6de43332bb0ffc4022bcfbce410f60008ef3878c0eeaea72c7552092e66\": container with ID starting with 1cfcf6de43332bb0ffc4022bcfbce410f60008ef3878c0eeaea72c7552092e66 not found: ID does not exist" containerID="1cfcf6de43332bb0ffc4022bcfbce410f60008ef3878c0eeaea72c7552092e66"
Sep 07 04:14:10.264619 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:10.264536    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cfcf6de43332bb0ffc4022bcfbce410f60008ef3878c0eeaea72c7552092e66"} err="failed to get container status \"1cfcf6de43332bb0ffc4022bcfbce410f60008ef3878c0eeaea72c7552092e66\": rpc error: code = NotFound desc = could not find container \"1cfcf6de43332bb0ffc4022bcfbce410f60008ef3878c0eeaea72c7552092e66\": container with ID starting with 1cfcf6de43332bb0ffc4022bcfbce410f60008ef3878c0eeaea72c7552092e66 not found: ID does not exist"
Sep 07 04:14:10.264619 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:10.264553    2583 scope.go:117] "RemoveContainer" containerID="49033437d7e36ee7a02631399af5fd9ced3e561bb771daebe2a846cd835abac0"
Sep 07 04:14:10.273115 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:10.273096    2583 scope.go:117] "RemoveContainer" containerID="49033437d7e36ee7a02631399af5fd9ced3e561bb771daebe2a846cd835abac0"
Sep 07 04:14:10.273412 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:14:10.273396    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49033437d7e36ee7a02631399af5fd9ced3e561bb771daebe2a846cd835abac0\": container with ID starting with 49033437d7e36ee7a02631399af5fd9ced3e561bb771daebe2a846cd835abac0 not found: ID does not exist" containerID="49033437d7e36ee7a02631399af5fd9ced3e561bb771daebe2a846cd835abac0"
Sep 07 04:14:10.273465 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:10.273420    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49033437d7e36ee7a02631399af5fd9ced3e561bb771daebe2a846cd835abac0"} err="failed to get container status \"49033437d7e36ee7a02631399af5fd9ced3e561bb771daebe2a846cd835abac0\": rpc error: code = NotFound desc = could not find container \"49033437d7e36ee7a02631399af5fd9ced3e561bb771daebe2a846cd835abac0\": container with ID starting with 49033437d7e36ee7a02631399af5fd9ced3e561bb771daebe2a846cd835abac0 not found: ID does not exist"
Sep 07 04:14:10.279677 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:10.279651    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"]
Sep 07 04:14:10.283087 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:10.283066    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c5ce6a-54c9fc74fc-cr4sf"]
Sep 07 04:14:10.292519 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:10.292498    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94"]
Sep 07 04:14:10.300485 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:10.300458    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c5ce6a-79cc675ff4-vgv94"]
Sep 07 04:14:11.522366 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:11.522334    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d6c8fe9-3a0a-4302-8328-9441009045ac" path="/var/lib/kubelet/pods/7d6c8fe9-3a0a-4302-8328-9441009045ac/volumes"
Sep 07 04:14:11.522730 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:11.522708    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9b38651-3b91-4da0-9e73-c432bf63f2ec" path="/var/lib/kubelet/pods/a9b38651-3b91-4da0-9e73-c432bf63f2ec/volumes"
Sep 07 04:14:15.850626 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:15.850597    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:14:15.911613 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:15.911579    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:14:16.151576 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:16.151494    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:14:20.284820 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:20.284772    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:14:35.524607 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:35.523789    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"]
Sep 07 04:14:35.524607 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:35.524024    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5" podUID="83cc94fd-1c50-4132-aa42-a2cb01445656" containerName="maas-api" containerID="cri-o://210b98c361776950f5052bc2226a33c9e44e81f61d71efdafdc431a097a4b67e" gracePeriod=30
Sep 07 04:14:35.551361 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:35.551335    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:14:35.637105 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:35.637080    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:14:35.705013 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:35.704972    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:14:35.756714 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:35.756682    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:14:35.773879 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:35.773857    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"
Sep 07 04:14:35.930939 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:35.930852    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83cc94fd-1c50-4132-aa42-a2cb01445656-maas-api-tls\") pod \"83cc94fd-1c50-4132-aa42-a2cb01445656\" (UID: \"83cc94fd-1c50-4132-aa42-a2cb01445656\") "
Sep 07 04:14:35.930939 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:35.930901    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83cc94fd-1c50-4132-aa42-a2cb01445656-metrics-certs\") pod \"83cc94fd-1c50-4132-aa42-a2cb01445656\" (UID: \"83cc94fd-1c50-4132-aa42-a2cb01445656\") "
Sep 07 04:14:35.931120 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:35.930953    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83cc94fd-1c50-4132-aa42-a2cb01445656-openshift-service-ca\") pod \"83cc94fd-1c50-4132-aa42-a2cb01445656\" (UID: \"83cc94fd-1c50-4132-aa42-a2cb01445656\") "
Sep 07 04:14:35.931162 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:35.931128    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7c2rr\" (UniqueName: \"kubernetes.io/projected/83cc94fd-1c50-4132-aa42-a2cb01445656-kube-api-access-7c2rr\") pod \"83cc94fd-1c50-4132-aa42-a2cb01445656\" (UID: \"83cc94fd-1c50-4132-aa42-a2cb01445656\") "
Sep 07 04:14:35.931303 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:35.931283    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83cc94fd-1c50-4132-aa42-a2cb01445656-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "83cc94fd-1c50-4132-aa42-a2cb01445656" (UID: "83cc94fd-1c50-4132-aa42-a2cb01445656"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 04:14:35.931513 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:35.931491    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83cc94fd-1c50-4132-aa42-a2cb01445656-openshift-service-ca\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:14:35.933189 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:35.933166    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83cc94fd-1c50-4132-aa42-a2cb01445656-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "83cc94fd-1c50-4132-aa42-a2cb01445656" (UID: "83cc94fd-1c50-4132-aa42-a2cb01445656"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:14:35.933302 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:35.933225    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83cc94fd-1c50-4132-aa42-a2cb01445656-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "83cc94fd-1c50-4132-aa42-a2cb01445656" (UID: "83cc94fd-1c50-4132-aa42-a2cb01445656"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:14:35.933351 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:35.933329    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83cc94fd-1c50-4132-aa42-a2cb01445656-kube-api-access-7c2rr" (OuterVolumeSpecName: "kube-api-access-7c2rr") pod "83cc94fd-1c50-4132-aa42-a2cb01445656" (UID: "83cc94fd-1c50-4132-aa42-a2cb01445656"). InnerVolumeSpecName "kube-api-access-7c2rr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:14:36.032879 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:36.032842    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83cc94fd-1c50-4132-aa42-a2cb01445656-maas-api-tls\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:14:36.032879 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:36.032871    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83cc94fd-1c50-4132-aa42-a2cb01445656-metrics-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:14:36.032879 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:36.032880    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7c2rr\" (UniqueName: \"kubernetes.io/projected/83cc94fd-1c50-4132-aa42-a2cb01445656-kube-api-access-7c2rr\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:14:36.162425 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:36.162388    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:14:36.394290 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:36.394248    2583 generic.go:356] "Generic (PLEG): container finished" podID="83cc94fd-1c50-4132-aa42-a2cb01445656" containerID="210b98c361776950f5052bc2226a33c9e44e81f61d71efdafdc431a097a4b67e" exitCode=0
Sep 07 04:14:36.394489 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:36.394298    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"
Sep 07 04:14:36.394489 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:36.394335    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5" event={"ID":"83cc94fd-1c50-4132-aa42-a2cb01445656","Type":"ContainerDied","Data":"210b98c361776950f5052bc2226a33c9e44e81f61d71efdafdc431a097a4b67e"}
Sep 07 04:14:36.394489 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:36.394409    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5" event={"ID":"83cc94fd-1c50-4132-aa42-a2cb01445656","Type":"ContainerDied","Data":"1aa32e5616a632a8655b11755e36f915a47b63d8c46372a8f153a7a8212ae200"}
Sep 07 04:14:36.394489 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:36.394433    2583 scope.go:117] "RemoveContainer" containerID="210b98c361776950f5052bc2226a33c9e44e81f61d71efdafdc431a097a4b67e"
Sep 07 04:14:36.404095 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:36.404077    2583 scope.go:117] "RemoveContainer" containerID="210b98c361776950f5052bc2226a33c9e44e81f61d71efdafdc431a097a4b67e"
Sep 07 04:14:36.404463 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:14:36.404441    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"210b98c361776950f5052bc2226a33c9e44e81f61d71efdafdc431a097a4b67e\": container with ID starting with 210b98c361776950f5052bc2226a33c9e44e81f61d71efdafdc431a097a4b67e not found: ID does not exist" containerID="210b98c361776950f5052bc2226a33c9e44e81f61d71efdafdc431a097a4b67e"
Sep 07 04:14:36.404532 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:36.404470    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"210b98c361776950f5052bc2226a33c9e44e81f61d71efdafdc431a097a4b67e"} err="failed to get container status \"210b98c361776950f5052bc2226a33c9e44e81f61d71efdafdc431a097a4b67e\": rpc error: code = NotFound desc = could not find container \"210b98c361776950f5052bc2226a33c9e44e81f61d71efdafdc431a097a4b67e\": container with ID starting with 210b98c361776950f5052bc2226a33c9e44e81f61d71efdafdc431a097a4b67e not found: ID does not exist"
Sep 07 04:14:36.414659 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:36.414634    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"]
Sep 07 04:14:36.418085 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:36.418062    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f8fdfa-5fc678bfc4-mmzx5"]
Sep 07 04:14:37.521755 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:37.521720    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83cc94fd-1c50-4132-aa42-a2cb01445656" path="/var/lib/kubelet/pods/83cc94fd-1c50-4132-aa42-a2cb01445656/volumes"
Sep 07 04:14:41.094149 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:41.094120    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"]
Sep 07 04:14:41.133557 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:41.133515    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:14:56.115329 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.115285    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw" podUID="b5c5ce30-34a0-4b11-a13c-0bdb60e98819" containerName="main" containerID="cri-o://40ccc4fbc15b560c3b1e94eab257221277a8f51bc53c1715ba3053691954155d" gracePeriod=45
Sep 07 04:14:56.361038 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.361009    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:14:56.406981 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.406893    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-tls-certs\") pod \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") "
Sep 07 04:14:56.406981 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.406968    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-home\") pod \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") "
Sep 07 04:14:56.407230 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.407009    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4jp7\" (UniqueName: \"kubernetes.io/projected/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-kube-api-access-k4jp7\") pod \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") "
Sep 07 04:14:56.407230 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.407026    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-model-cache\") pod \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") "
Sep 07 04:14:56.407230 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.407048    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-dshm\") pod \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") "
Sep 07 04:14:56.407230 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.407095    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-tmp-dir\") pod \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\" (UID: \"b5c5ce30-34a0-4b11-a13c-0bdb60e98819\") "
Sep 07 04:14:56.407444 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.407317    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-home" (OuterVolumeSpecName: "home") pod "b5c5ce30-34a0-4b11-a13c-0bdb60e98819" (UID: "b5c5ce30-34a0-4b11-a13c-0bdb60e98819"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:14:56.407444 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.407330    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-model-cache" (OuterVolumeSpecName: "model-cache") pod "b5c5ce30-34a0-4b11-a13c-0bdb60e98819" (UID: "b5c5ce30-34a0-4b11-a13c-0bdb60e98819"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:14:56.407444 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.407420    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-home\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:14:56.407444 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.407438    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-model-cache\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:14:56.407618 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.407505    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b5c5ce30-34a0-4b11-a13c-0bdb60e98819" (UID: "b5c5ce30-34a0-4b11-a13c-0bdb60e98819"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:14:56.409567 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.409535    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-kube-api-access-k4jp7" (OuterVolumeSpecName: "kube-api-access-k4jp7") pod "b5c5ce30-34a0-4b11-a13c-0bdb60e98819" (UID: "b5c5ce30-34a0-4b11-a13c-0bdb60e98819"). InnerVolumeSpecName "kube-api-access-k4jp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:14:56.409738 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.409722    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-dshm" (OuterVolumeSpecName: "dshm") pod "b5c5ce30-34a0-4b11-a13c-0bdb60e98819" (UID: "b5c5ce30-34a0-4b11-a13c-0bdb60e98819"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:14:56.409962 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.409935    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b5c5ce30-34a0-4b11-a13c-0bdb60e98819" (UID: "b5c5ce30-34a0-4b11-a13c-0bdb60e98819"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:14:56.493871 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.493832    2583 generic.go:356] "Generic (PLEG): container finished" podID="b5c5ce30-34a0-4b11-a13c-0bdb60e98819" containerID="40ccc4fbc15b560c3b1e94eab257221277a8f51bc53c1715ba3053691954155d" exitCode=0
Sep 07 04:14:56.493871 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.493876    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"
Sep 07 04:14:56.494129 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.493918    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw" event={"ID":"b5c5ce30-34a0-4b11-a13c-0bdb60e98819","Type":"ContainerDied","Data":"40ccc4fbc15b560c3b1e94eab257221277a8f51bc53c1715ba3053691954155d"}
Sep 07 04:14:56.494129 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.493972    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw" event={"ID":"b5c5ce30-34a0-4b11-a13c-0bdb60e98819","Type":"ContainerDied","Data":"311c9db29e8bdceb1b848ad05fbe9584caa741b62f3132c0d67e38ef0f7f7db9"}
Sep 07 04:14:56.494129 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.493997    2583 scope.go:117] "RemoveContainer" containerID="40ccc4fbc15b560c3b1e94eab257221277a8f51bc53c1715ba3053691954155d"
Sep 07 04:14:56.506361 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.506331    2583 scope.go:117] "RemoveContainer" containerID="40ccc4fbc15b560c3b1e94eab257221277a8f51bc53c1715ba3053691954155d"
Sep 07 04:14:56.506771 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:14:56.506740    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40ccc4fbc15b560c3b1e94eab257221277a8f51bc53c1715ba3053691954155d\": container with ID starting with 40ccc4fbc15b560c3b1e94eab257221277a8f51bc53c1715ba3053691954155d not found: ID does not exist" containerID="40ccc4fbc15b560c3b1e94eab257221277a8f51bc53c1715ba3053691954155d"
Sep 07 04:14:56.506894 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.506781    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40ccc4fbc15b560c3b1e94eab257221277a8f51bc53c1715ba3053691954155d"} err="failed to get container status \"40ccc4fbc15b560c3b1e94eab257221277a8f51bc53c1715ba3053691954155d\": rpc error: code = NotFound desc = could not find container \"40ccc4fbc15b560c3b1e94eab257221277a8f51bc53c1715ba3053691954155d\": container with ID starting with 40ccc4fbc15b560c3b1e94eab257221277a8f51bc53c1715ba3053691954155d not found: ID does not exist"
Sep 07 04:14:56.510281 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.508716    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-tmp-dir\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:14:56.510281 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.508761    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-tls-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:14:56.510281 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.508776    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4jp7\" (UniqueName: \"kubernetes.io/projected/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-kube-api-access-k4jp7\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:14:56.510281 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.508798    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5c5ce30-34a0-4b11-a13c-0bdb60e98819-dshm\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:14:56.517341 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.517297    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"]
Sep 07 04:14:56.519228 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:56.519194    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-f8fdfa/ipp-route-f8fdfa-kserve-7545cf7d7f-q2pdw"]
Sep 07 04:14:57.521691 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:14:57.521652    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5c5ce30-34a0-4b11-a13c-0bdb60e98819" path="/var/lib/kubelet/pods/b5c5ce30-34a0-4b11-a13c-0bdb60e98819/volumes"
Sep 07 04:15:00.139911 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:00.139873    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-704b16-29812575-2dxh5"]
Sep 07 04:15:00.140348 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:00.140296    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d6c8fe9-3a0a-4302-8328-9441009045ac" containerName="payload-pre-processing"
Sep 07 04:15:00.140348 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:00.140309    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d6c8fe9-3a0a-4302-8328-9441009045ac" containerName="payload-pre-processing"
Sep 07 04:15:00.140348 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:00.140320    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83cc94fd-1c50-4132-aa42-a2cb01445656" containerName="maas-api"
Sep 07 04:15:00.140348 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:00.140325    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="83cc94fd-1c50-4132-aa42-a2cb01445656" containerName="maas-api"
Sep 07 04:15:00.140348 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:00.140344    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9b38651-3b91-4da0-9e73-c432bf63f2ec" containerName="maas-api"
Sep 07 04:15:00.140348 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:00.140350    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9b38651-3b91-4da0-9e73-c432bf63f2ec" containerName="maas-api"
Sep 07 04:15:00.140568 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:00.140358    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5c5ce30-34a0-4b11-a13c-0bdb60e98819" containerName="main"
Sep 07 04:15:00.140568 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:00.140363    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5c5ce30-34a0-4b11-a13c-0bdb60e98819" containerName="main"
Sep 07 04:15:00.140568 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:00.140416    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d6c8fe9-3a0a-4302-8328-9441009045ac" containerName="payload-pre-processing"
Sep 07 04:15:00.140568 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:00.140427    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5c5ce30-34a0-4b11-a13c-0bdb60e98819" containerName="main"
Sep 07 04:15:00.140568 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:00.140433    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="83cc94fd-1c50-4132-aa42-a2cb01445656" containerName="maas-api"
Sep 07 04:15:00.140568 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:00.140439    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9b38651-3b91-4da0-9e73-c432bf63f2ec" containerName="maas-api"
Sep 07 04:15:00.144913 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:00.144895    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-704b16-29812575-2dxh5"
Sep 07 04:15:00.147014 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:00.146995    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zs4ff\""
Sep 07 04:15:00.152467 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:00.152439    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-704b16-29812575-2dxh5"]
Sep 07 04:15:00.157080 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:00.157054    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-704b16-29812575-2dxh5"
Sep 07 04:15:00.435699 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:00.435662    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-704b16-29812575-2dxh5"]
Sep 07 04:15:00.437825 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:15:00.437801    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00ff1d24_5e49_4d55_b210_6f9dd83f3d43.slice/crio-2ab7cfaa2b6a08f1ca270d46c16a5bb53ec72ed7d3d132dd378958fefbfbc0c8 WatchSource:0}: Error finding container 2ab7cfaa2b6a08f1ca270d46c16a5bb53ec72ed7d3d132dd378958fefbfbc0c8: Status 404 returned error can't find the container with id 2ab7cfaa2b6a08f1ca270d46c16a5bb53ec72ed7d3d132dd378958fefbfbc0c8
Sep 07 04:15:00.516809 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:00.516774    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-704b16-29812575-2dxh5" event={"ID":"00ff1d24-5e49-4d55-b210-6f9dd83f3d43","Type":"ContainerStarted","Data":"2ab7cfaa2b6a08f1ca270d46c16a5bb53ec72ed7d3d132dd378958fefbfbc0c8"}
Sep 07 04:15:05.543356 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:05.543316    2583 generic.go:356] "Generic (PLEG): container finished" podID="00ff1d24-5e49-4d55-b210-6f9dd83f3d43" containerID="577f54c14cb406211783f2c408f85bf2b4755d32d8afee3cee4a4d04810c3516" exitCode=0
Sep 07 04:15:05.543811 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:05.543378    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-704b16-29812575-2dxh5" event={"ID":"00ff1d24-5e49-4d55-b210-6f9dd83f3d43","Type":"ContainerDied","Data":"577f54c14cb406211783f2c408f85bf2b4755d32d8afee3cee4a4d04810c3516"}
Sep 07 04:15:06.461181 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:06.461145    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:06.681058 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:06.681034    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-704b16-29812575-2dxh5"
Sep 07 04:15:07.554042 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:07.554016    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-704b16-29812575-2dxh5"
Sep 07 04:15:07.554199 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:07.554049    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-704b16-29812575-2dxh5" event={"ID":"00ff1d24-5e49-4d55-b210-6f9dd83f3d43","Type":"ContainerDied","Data":"2ab7cfaa2b6a08f1ca270d46c16a5bb53ec72ed7d3d132dd378958fefbfbc0c8"}
Sep 07 04:15:07.554199 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:07.554094    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ab7cfaa2b6a08f1ca270d46c16a5bb53ec72ed7d3d132dd378958fefbfbc0c8"
Sep 07 04:15:17.819825 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:17.819793    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:17.993770 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:17.993737    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:23.606049 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:23.606011    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:29.404230 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:29.404138    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:29.504511 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:29.504481    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:29.683070 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:29.682974    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:30.194788 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:30.194748    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:34.987748 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:34.987711    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:40.293319 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:40.293277    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:40.790165 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:40.790136    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:40.797470 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:40.797437    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-704b16-29812575-2dxh5"]
Sep 07 04:15:40.801070 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:40.801040    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-704b16-29812575-2dxh5"]
Sep 07 04:15:40.827736 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:40.827705    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj"]
Sep 07 04:15:40.828285 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:40.828206    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj" podUID="3003c51a-85fe-47d9-87b5-4639378d85ce" containerName="payload-pre-processing" containerID="cri-o://744a8ef9648ad3cc156b3183dfc31662a7933c01ffc27ea118c2ae5297e59046" gracePeriod=30
Sep 07 04:15:40.889665 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:40.889636    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:40.953763 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:40.953734    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:41.007833 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:41.007796    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:41.079670 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:41.079648    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj"
Sep 07 04:15:41.143458 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:41.143430    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmdbv\" (UniqueName: \"kubernetes.io/projected/3003c51a-85fe-47d9-87b5-4639378d85ce-kube-api-access-mmdbv\") pod \"3003c51a-85fe-47d9-87b5-4639378d85ce\" (UID: \"3003c51a-85fe-47d9-87b5-4639378d85ce\") "
Sep 07 04:15:41.143628 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:41.143476    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3003c51a-85fe-47d9-87b5-4639378d85ce-plugins-config-volume\") pod \"3003c51a-85fe-47d9-87b5-4639378d85ce\" (UID: \"3003c51a-85fe-47d9-87b5-4639378d85ce\") "
Sep 07 04:15:41.143894 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:41.143862    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3003c51a-85fe-47d9-87b5-4639378d85ce-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3003c51a-85fe-47d9-87b5-4639378d85ce" (UID: "3003c51a-85fe-47d9-87b5-4639378d85ce"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 04:15:41.145764 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:41.145743    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3003c51a-85fe-47d9-87b5-4639378d85ce-kube-api-access-mmdbv" (OuterVolumeSpecName: "kube-api-access-mmdbv") pod "3003c51a-85fe-47d9-87b5-4639378d85ce" (UID: "3003c51a-85fe-47d9-87b5-4639378d85ce"). InnerVolumeSpecName "kube-api-access-mmdbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:15:41.245123 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:41.245086    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmdbv\" (UniqueName: \"kubernetes.io/projected/3003c51a-85fe-47d9-87b5-4639378d85ce-kube-api-access-mmdbv\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:15:41.245123 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:41.245117    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3003c51a-85fe-47d9-87b5-4639378d85ce-plugins-config-volume\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:15:41.328538 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:41.328429    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:41.522417 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:41.522377    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00ff1d24-5e49-4d55-b210-6f9dd83f3d43" path="/var/lib/kubelet/pods/00ff1d24-5e49-4d55-b210-6f9dd83f3d43/volumes"
Sep 07 04:15:41.722926 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:41.722887    2583 generic.go:356] "Generic (PLEG): container finished" podID="3003c51a-85fe-47d9-87b5-4639378d85ce" containerID="744a8ef9648ad3cc156b3183dfc31662a7933c01ffc27ea118c2ae5297e59046" exitCode=0
Sep 07 04:15:41.723136 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:41.722957    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj"
Sep 07 04:15:41.723136 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:41.722991    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj" event={"ID":"3003c51a-85fe-47d9-87b5-4639378d85ce","Type":"ContainerDied","Data":"744a8ef9648ad3cc156b3183dfc31662a7933c01ffc27ea118c2ae5297e59046"}
Sep 07 04:15:41.723136 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:41.723073    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj" event={"ID":"3003c51a-85fe-47d9-87b5-4639378d85ce","Type":"ContainerDied","Data":"22aa76a25a3b848b120a7a4c724e5f052d898ae6fd1e228cbe41dc921fa8ae2e"}
Sep 07 04:15:41.723136 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:41.723104    2583 scope.go:117] "RemoveContainer" containerID="744a8ef9648ad3cc156b3183dfc31662a7933c01ffc27ea118c2ae5297e59046"
Sep 07 04:15:41.733500 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:41.733479    2583 scope.go:117] "RemoveContainer" containerID="744a8ef9648ad3cc156b3183dfc31662a7933c01ffc27ea118c2ae5297e59046"
Sep 07 04:15:41.733785 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:15:41.733765    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"744a8ef9648ad3cc156b3183dfc31662a7933c01ffc27ea118c2ae5297e59046\": container with ID starting with 744a8ef9648ad3cc156b3183dfc31662a7933c01ffc27ea118c2ae5297e59046 not found: ID does not exist" containerID="744a8ef9648ad3cc156b3183dfc31662a7933c01ffc27ea118c2ae5297e59046"
Sep 07 04:15:41.733829 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:41.733795    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"744a8ef9648ad3cc156b3183dfc31662a7933c01ffc27ea118c2ae5297e59046"} err="failed to get container status \"744a8ef9648ad3cc156b3183dfc31662a7933c01ffc27ea118c2ae5297e59046\": rpc error: code = NotFound desc = could not find container \"744a8ef9648ad3cc156b3183dfc31662a7933c01ffc27ea118c2ae5297e59046\": container with ID starting with 744a8ef9648ad3cc156b3183dfc31662a7933c01ffc27ea118c2ae5297e59046 not found: ID does not exist"
Sep 07 04:15:41.742464 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:41.742436    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj"]
Sep 07 04:15:41.747296 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:41.747274    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-704b16-766c48669d-wmwlj"]
Sep 07 04:15:43.521785 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:43.521748    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3003c51a-85fe-47d9-87b5-4639378d85ce" path="/var/lib/kubelet/pods/3003c51a-85fe-47d9-87b5-4639378d85ce/volumes"
Sep 07 04:15:47.083093 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:47.083058    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:47.148647 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:47.148620    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:47.347966 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:47.347887    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:47.796058 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:47.796020    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:58.038529 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.038494    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"]
Sep 07 04:15:58.038991 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.038744    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft" podUID="1ae9093c-0ed4-476d-abde-ccaed0cf2ad8" containerName="maas-api" containerID="cri-o://c5f509b8804b7edd6f7ddf22a52bff2e032e2843412d76fbba6c3cfaf32e45a8" gracePeriod=30
Sep 07 04:15:58.070473 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.070435    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:58.100589 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.100554    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74"]
Sep 07 04:15:58.100816 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.100790    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74" podUID="80646e0f-ef88-407d-b373-2e84d7517194" containerName="payload-processing" containerID="cri-o://460d83d363b57c0aa21e87151c60f611b88473fc604af7aa19267380c82089b4" gracePeriod=30
Sep 07 04:15:58.164976 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.164945    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:58.240512 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.240476    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:58.302779 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.302696    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:58.354487 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.354466    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"
Sep 07 04:15:58.383939 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.383916    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74"
Sep 07 04:15:58.501986 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.501940    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-maas-api-tls\") pod \"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8\" (UID: \"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8\") "
Sep 07 04:15:58.501986 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.501997    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-metrics-certs\") pod \"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8\" (UID: \"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8\") "
Sep 07 04:15:58.502307 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.502033    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-openshift-service-ca\") pod \"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8\" (UID: \"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8\") "
Sep 07 04:15:58.502307 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.502055    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfql9\" (UniqueName: \"kubernetes.io/projected/80646e0f-ef88-407d-b373-2e84d7517194-kube-api-access-bfql9\") pod \"80646e0f-ef88-407d-b373-2e84d7517194\" (UID: \"80646e0f-ef88-407d-b373-2e84d7517194\") "
Sep 07 04:15:58.502307 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.502130    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80646e0f-ef88-407d-b373-2e84d7517194-plugins-config-volume\") pod \"80646e0f-ef88-407d-b373-2e84d7517194\" (UID: \"80646e0f-ef88-407d-b373-2e84d7517194\") "
Sep 07 04:15:58.502307 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.502171    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9jds\" (UniqueName: \"kubernetes.io/projected/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-kube-api-access-b9jds\") pod \"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8\" (UID: \"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8\") "
Sep 07 04:15:58.502542 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.502470    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1ae9093c-0ed4-476d-abde-ccaed0cf2ad8" (UID: "1ae9093c-0ed4-476d-abde-ccaed0cf2ad8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 04:15:58.502621 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.502593    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80646e0f-ef88-407d-b373-2e84d7517194-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "80646e0f-ef88-407d-b373-2e84d7517194" (UID: "80646e0f-ef88-407d-b373-2e84d7517194"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 04:15:58.504487 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.504464    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1ae9093c-0ed4-476d-abde-ccaed0cf2ad8" (UID: "1ae9093c-0ed4-476d-abde-ccaed0cf2ad8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:15:58.504487 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.504470    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-kube-api-access-b9jds" (OuterVolumeSpecName: "kube-api-access-b9jds") pod "1ae9093c-0ed4-476d-abde-ccaed0cf2ad8" (UID: "1ae9093c-0ed4-476d-abde-ccaed0cf2ad8"). InnerVolumeSpecName "kube-api-access-b9jds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:15:58.504821 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.504801    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80646e0f-ef88-407d-b373-2e84d7517194-kube-api-access-bfql9" (OuterVolumeSpecName: "kube-api-access-bfql9") pod "80646e0f-ef88-407d-b373-2e84d7517194" (UID: "80646e0f-ef88-407d-b373-2e84d7517194"). InnerVolumeSpecName "kube-api-access-bfql9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:15:58.504821 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.504803    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1ae9093c-0ed4-476d-abde-ccaed0cf2ad8" (UID: "1ae9093c-0ed4-476d-abde-ccaed0cf2ad8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:15:58.603782 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.603688    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-openshift-service-ca\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:15:58.603782 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.603712    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfql9\" (UniqueName: \"kubernetes.io/projected/80646e0f-ef88-407d-b373-2e84d7517194-kube-api-access-bfql9\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:15:58.603782 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.603727    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80646e0f-ef88-407d-b373-2e84d7517194-plugins-config-volume\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:15:58.603782 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.603742    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9jds\" (UniqueName: \"kubernetes.io/projected/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-kube-api-access-b9jds\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:15:58.603782 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.603751    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-maas-api-tls\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:15:58.603782 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.603759    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8-metrics-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:15:58.622539 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.622501    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:15:58.804698 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.804666    2583 generic.go:356] "Generic (PLEG): container finished" podID="1ae9093c-0ed4-476d-abde-ccaed0cf2ad8" containerID="c5f509b8804b7edd6f7ddf22a52bff2e032e2843412d76fbba6c3cfaf32e45a8" exitCode=0
Sep 07 04:15:58.804896 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.804715    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"
Sep 07 04:15:58.804896 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.804755    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft" event={"ID":"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8","Type":"ContainerDied","Data":"c5f509b8804b7edd6f7ddf22a52bff2e032e2843412d76fbba6c3cfaf32e45a8"}
Sep 07 04:15:58.804896 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.804815    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft" event={"ID":"1ae9093c-0ed4-476d-abde-ccaed0cf2ad8","Type":"ContainerDied","Data":"77494320a49bff0e04c7a5e2b2a3968bf01de6ba5d5ed4cc14fb55505a800bf7"}
Sep 07 04:15:58.804896 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.804843    2583 scope.go:117] "RemoveContainer" containerID="c5f509b8804b7edd6f7ddf22a52bff2e032e2843412d76fbba6c3cfaf32e45a8"
Sep 07 04:15:58.806117 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.806075    2583 generic.go:356] "Generic (PLEG): container finished" podID="80646e0f-ef88-407d-b373-2e84d7517194" containerID="460d83d363b57c0aa21e87151c60f611b88473fc604af7aa19267380c82089b4" exitCode=0
Sep 07 04:15:58.806197 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.806126    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74"
Sep 07 04:15:58.806197 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.806128    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74" event={"ID":"80646e0f-ef88-407d-b373-2e84d7517194","Type":"ContainerDied","Data":"460d83d363b57c0aa21e87151c60f611b88473fc604af7aa19267380c82089b4"}
Sep 07 04:15:58.806197 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.806169    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74" event={"ID":"80646e0f-ef88-407d-b373-2e84d7517194","Type":"ContainerDied","Data":"8e6e178729eb58207eff752894d475c957a07b5722b28eb7742d84b79e3dd224"}
Sep 07 04:15:58.814449 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.814422    2583 scope.go:117] "RemoveContainer" containerID="c5f509b8804b7edd6f7ddf22a52bff2e032e2843412d76fbba6c3cfaf32e45a8"
Sep 07 04:15:58.814773 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:15:58.814753    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5f509b8804b7edd6f7ddf22a52bff2e032e2843412d76fbba6c3cfaf32e45a8\": container with ID starting with c5f509b8804b7edd6f7ddf22a52bff2e032e2843412d76fbba6c3cfaf32e45a8 not found: ID does not exist" containerID="c5f509b8804b7edd6f7ddf22a52bff2e032e2843412d76fbba6c3cfaf32e45a8"
Sep 07 04:15:58.814888 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.814782    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5f509b8804b7edd6f7ddf22a52bff2e032e2843412d76fbba6c3cfaf32e45a8"} err="failed to get container status \"c5f509b8804b7edd6f7ddf22a52bff2e032e2843412d76fbba6c3cfaf32e45a8\": rpc error: code = NotFound desc = could not find container \"c5f509b8804b7edd6f7ddf22a52bff2e032e2843412d76fbba6c3cfaf32e45a8\": container with ID starting with c5f509b8804b7edd6f7ddf22a52bff2e032e2843412d76fbba6c3cfaf32e45a8 not found: ID does not exist"
Sep 07 04:15:58.814888 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.814806    2583 scope.go:117] "RemoveContainer" containerID="460d83d363b57c0aa21e87151c60f611b88473fc604af7aa19267380c82089b4"
Sep 07 04:15:58.823338 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.823314    2583 scope.go:117] "RemoveContainer" containerID="460d83d363b57c0aa21e87151c60f611b88473fc604af7aa19267380c82089b4"
Sep 07 04:15:58.823633 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:15:58.823616    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"460d83d363b57c0aa21e87151c60f611b88473fc604af7aa19267380c82089b4\": container with ID starting with 460d83d363b57c0aa21e87151c60f611b88473fc604af7aa19267380c82089b4 not found: ID does not exist" containerID="460d83d363b57c0aa21e87151c60f611b88473fc604af7aa19267380c82089b4"
Sep 07 04:15:58.823675 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.823642    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"460d83d363b57c0aa21e87151c60f611b88473fc604af7aa19267380c82089b4"} err="failed to get container status \"460d83d363b57c0aa21e87151c60f611b88473fc604af7aa19267380c82089b4\": rpc error: code = NotFound desc = could not find container \"460d83d363b57c0aa21e87151c60f611b88473fc604af7aa19267380c82089b4\": container with ID starting with 460d83d363b57c0aa21e87151c60f611b88473fc604af7aa19267380c82089b4 not found: ID does not exist"
Sep 07 04:15:58.828811 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.828788    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"]
Sep 07 04:15:58.832438 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.832417    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b8cea6-86b4558cdc-j9nft"]
Sep 07 04:15:58.841989 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.841969    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74"]
Sep 07 04:15:58.847617 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:58.847596    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b8cea6-f57bcddfc-4lp74"]
Sep 07 04:15:59.528968 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:59.528939    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ae9093c-0ed4-476d-abde-ccaed0cf2ad8" path="/var/lib/kubelet/pods/1ae9093c-0ed4-476d-abde-ccaed0cf2ad8/volumes"
Sep 07 04:15:59.529405 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:15:59.529299    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80646e0f-ef88-407d-b373-2e84d7517194" path="/var/lib/kubelet/pods/80646e0f-ef88-407d-b373-2e84d7517194/volumes"
Sep 07 04:16:04.570043 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:04.570012    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:16:04.622789 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:04.622758    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:16:04.834196 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:04.834122    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:16:05.272035 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:05.272001    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:16:15.561701 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:15.561667    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:16:15.619050 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:15.619012    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:16:15.711756 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:15.711705    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:16:15.764596 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:15.764562    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:16:16.112248 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:16.112199    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:16:21.947421 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:21.947386    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:16:22.010979 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:22.010946    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:16:22.242055 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:22.241972    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:16:24.969199 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:24.969163    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-9f656d559-wbsst"]
Sep 07 04:16:24.969703 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:24.969398    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-9f656d559-wbsst" podUID="60352d1c-5ce1-4acd-a70c-befba153463a" containerName="manager" containerID="cri-o://ca381ec4231fb99cceb81c4d7b32a13a658a8c5a77495d44bed5fef4f5706a7a" gracePeriod=30
Sep 07 04:16:25.215972 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:25.215952    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9f656d559-wbsst"
Sep 07 04:16:25.341769 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:25.341675    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rhs6\" (UniqueName: \"kubernetes.io/projected/60352d1c-5ce1-4acd-a70c-befba153463a-kube-api-access-6rhs6\") pod \"60352d1c-5ce1-4acd-a70c-befba153463a\" (UID: \"60352d1c-5ce1-4acd-a70c-befba153463a\") "
Sep 07 04:16:25.341920 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:25.341789    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60352d1c-5ce1-4acd-a70c-befba153463a-webhook-cert\") pod \"60352d1c-5ce1-4acd-a70c-befba153463a\" (UID: \"60352d1c-5ce1-4acd-a70c-befba153463a\") "
Sep 07 04:16:25.341920 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:25.341810    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60352d1c-5ce1-4acd-a70c-befba153463a-metrics-certs\") pod \"60352d1c-5ce1-4acd-a70c-befba153463a\" (UID: \"60352d1c-5ce1-4acd-a70c-befba153463a\") "
Sep 07 04:16:25.343967 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:25.343931    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60352d1c-5ce1-4acd-a70c-befba153463a-kube-api-access-6rhs6" (OuterVolumeSpecName: "kube-api-access-6rhs6") pod "60352d1c-5ce1-4acd-a70c-befba153463a" (UID: "60352d1c-5ce1-4acd-a70c-befba153463a"). InnerVolumeSpecName "kube-api-access-6rhs6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:16:25.344110 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:25.343983    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60352d1c-5ce1-4acd-a70c-befba153463a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "60352d1c-5ce1-4acd-a70c-befba153463a" (UID: "60352d1c-5ce1-4acd-a70c-befba153463a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:16:25.344110 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:25.343999    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60352d1c-5ce1-4acd-a70c-befba153463a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "60352d1c-5ce1-4acd-a70c-befba153463a" (UID: "60352d1c-5ce1-4acd-a70c-befba153463a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:16:25.443110 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:25.443064    2583 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60352d1c-5ce1-4acd-a70c-befba153463a-webhook-cert\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:16:25.443110 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:25.443105    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60352d1c-5ce1-4acd-a70c-befba153463a-metrics-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:16:25.443110 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:25.443114    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rhs6\" (UniqueName: \"kubernetes.io/projected/60352d1c-5ce1-4acd-a70c-befba153463a-kube-api-access-6rhs6\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:16:25.934378 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:25.934337    2583 generic.go:356] "Generic (PLEG): container finished" podID="60352d1c-5ce1-4acd-a70c-befba153463a" containerID="ca381ec4231fb99cceb81c4d7b32a13a658a8c5a77495d44bed5fef4f5706a7a" exitCode=0
Sep 07 04:16:25.934565 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:25.934390    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9f656d559-wbsst"
Sep 07 04:16:25.934565 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:25.934428    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9f656d559-wbsst" event={"ID":"60352d1c-5ce1-4acd-a70c-befba153463a","Type":"ContainerDied","Data":"ca381ec4231fb99cceb81c4d7b32a13a658a8c5a77495d44bed5fef4f5706a7a"}
Sep 07 04:16:25.934565 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:25.934509    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9f656d559-wbsst" event={"ID":"60352d1c-5ce1-4acd-a70c-befba153463a","Type":"ContainerDied","Data":"9b847dd76d44952b37ea8a5a888962fe1b4ded3b9ca7b5f4a4100a99ee98e217"}
Sep 07 04:16:25.934565 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:25.934532    2583 scope.go:117] "RemoveContainer" containerID="ca381ec4231fb99cceb81c4d7b32a13a658a8c5a77495d44bed5fef4f5706a7a"
Sep 07 04:16:25.943718 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:25.943690    2583 scope.go:117] "RemoveContainer" containerID="ca381ec4231fb99cceb81c4d7b32a13a658a8c5a77495d44bed5fef4f5706a7a"
Sep 07 04:16:25.944016 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:16:25.943997    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca381ec4231fb99cceb81c4d7b32a13a658a8c5a77495d44bed5fef4f5706a7a\": container with ID starting with ca381ec4231fb99cceb81c4d7b32a13a658a8c5a77495d44bed5fef4f5706a7a not found: ID does not exist" containerID="ca381ec4231fb99cceb81c4d7b32a13a658a8c5a77495d44bed5fef4f5706a7a"
Sep 07 04:16:25.944067 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:25.944025    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca381ec4231fb99cceb81c4d7b32a13a658a8c5a77495d44bed5fef4f5706a7a"} err="failed to get container status \"ca381ec4231fb99cceb81c4d7b32a13a658a8c5a77495d44bed5fef4f5706a7a\": rpc error: code = NotFound desc = could not find container \"ca381ec4231fb99cceb81c4d7b32a13a658a8c5a77495d44bed5fef4f5706a7a\": container with ID starting with ca381ec4231fb99cceb81c4d7b32a13a658a8c5a77495d44bed5fef4f5706a7a not found: ID does not exist"
Sep 07 04:16:25.950272 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:25.950247    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-9f656d559-wbsst"]
Sep 07 04:16:25.953628 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:25.953606    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-9f656d559-wbsst"]
Sep 07 04:16:27.521910 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:16:27.521877    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60352d1c-5ce1-4acd-a70c-befba153463a" path="/var/lib/kubelet/pods/60352d1c-5ce1-4acd-a70c-befba153463a/volumes"
Sep 07 04:17:24.618753 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:24.618715    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:17:26.810915 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.810881    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-9f656d559-798gc"]
Sep 07 04:17:26.811454 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.811268    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ae9093c-0ed4-476d-abde-ccaed0cf2ad8" containerName="maas-api"
Sep 07 04:17:26.811454 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.811281    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ae9093c-0ed4-476d-abde-ccaed0cf2ad8" containerName="maas-api"
Sep 07 04:17:26.811454 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.811293    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60352d1c-5ce1-4acd-a70c-befba153463a" containerName="manager"
Sep 07 04:17:26.811454 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.811299    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="60352d1c-5ce1-4acd-a70c-befba153463a" containerName="manager"
Sep 07 04:17:26.811454 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.811310    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80646e0f-ef88-407d-b373-2e84d7517194" containerName="payload-processing"
Sep 07 04:17:26.811454 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.811315    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="80646e0f-ef88-407d-b373-2e84d7517194" containerName="payload-processing"
Sep 07 04:17:26.811454 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.811322    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3003c51a-85fe-47d9-87b5-4639378d85ce" containerName="payload-pre-processing"
Sep 07 04:17:26.811454 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.811327    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3003c51a-85fe-47d9-87b5-4639378d85ce" containerName="payload-pre-processing"
Sep 07 04:17:26.811454 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.811335    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00ff1d24-5e49-4d55-b210-6f9dd83f3d43" containerName="cleanup"
Sep 07 04:17:26.811454 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.811341    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="00ff1d24-5e49-4d55-b210-6f9dd83f3d43" containerName="cleanup"
Sep 07 04:17:26.811454 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.811399    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ae9093c-0ed4-476d-abde-ccaed0cf2ad8" containerName="maas-api"
Sep 07 04:17:26.811454 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.811407    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="60352d1c-5ce1-4acd-a70c-befba153463a" containerName="manager"
Sep 07 04:17:26.811454 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.811413    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3003c51a-85fe-47d9-87b5-4639378d85ce" containerName="payload-pre-processing"
Sep 07 04:17:26.811454 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.811419    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="00ff1d24-5e49-4d55-b210-6f9dd83f3d43" containerName="cleanup"
Sep 07 04:17:26.811454 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.811424    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="80646e0f-ef88-407d-b373-2e84d7517194" containerName="payload-processing"
Sep 07 04:17:26.813271 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.813257    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9f656d559-798gc"
Sep 07 04:17:26.815533 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.815510    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 07 04:17:26.816559 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.816534    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 07 04:17:26.816663 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.816564    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pl7hr\""
Sep 07 04:17:26.816731 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.816684    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 07 04:17:26.823793 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.823771    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9f656d559-798gc"]
Sep 07 04:17:26.905985 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.905946    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4615aac9-3661-4053-81cc-deaead1ad409-metrics-certs\") pod \"maas-controller-9f656d559-798gc\" (UID: \"4615aac9-3661-4053-81cc-deaead1ad409\") " pod="opendatahub/maas-controller-9f656d559-798gc"
Sep 07 04:17:26.906140 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.906039    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4615aac9-3661-4053-81cc-deaead1ad409-webhook-cert\") pod \"maas-controller-9f656d559-798gc\" (UID: \"4615aac9-3661-4053-81cc-deaead1ad409\") " pod="opendatahub/maas-controller-9f656d559-798gc"
Sep 07 04:17:26.906140 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:26.906064    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68mrz\" (UniqueName: \"kubernetes.io/projected/4615aac9-3661-4053-81cc-deaead1ad409-kube-api-access-68mrz\") pod \"maas-controller-9f656d559-798gc\" (UID: \"4615aac9-3661-4053-81cc-deaead1ad409\") " pod="opendatahub/maas-controller-9f656d559-798gc"
Sep 07 04:17:27.006934 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:27.006891    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4615aac9-3661-4053-81cc-deaead1ad409-webhook-cert\") pod \"maas-controller-9f656d559-798gc\" (UID: \"4615aac9-3661-4053-81cc-deaead1ad409\") " pod="opendatahub/maas-controller-9f656d559-798gc"
Sep 07 04:17:27.007143 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:27.006943    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68mrz\" (UniqueName: \"kubernetes.io/projected/4615aac9-3661-4053-81cc-deaead1ad409-kube-api-access-68mrz\") pod \"maas-controller-9f656d559-798gc\" (UID: \"4615aac9-3661-4053-81cc-deaead1ad409\") " pod="opendatahub/maas-controller-9f656d559-798gc"
Sep 07 04:17:27.007143 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:27.006992    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4615aac9-3661-4053-81cc-deaead1ad409-metrics-certs\") pod \"maas-controller-9f656d559-798gc\" (UID: \"4615aac9-3661-4053-81cc-deaead1ad409\") " pod="opendatahub/maas-controller-9f656d559-798gc"
Sep 07 04:17:27.009558 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:27.009533    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4615aac9-3661-4053-81cc-deaead1ad409-webhook-cert\") pod \"maas-controller-9f656d559-798gc\" (UID: \"4615aac9-3661-4053-81cc-deaead1ad409\") " pod="opendatahub/maas-controller-9f656d559-798gc"
Sep 07 04:17:27.009666 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:27.009570    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4615aac9-3661-4053-81cc-deaead1ad409-metrics-certs\") pod \"maas-controller-9f656d559-798gc\" (UID: \"4615aac9-3661-4053-81cc-deaead1ad409\") " pod="opendatahub/maas-controller-9f656d559-798gc"
Sep 07 04:17:27.015934 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:27.015904    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68mrz\" (UniqueName: \"kubernetes.io/projected/4615aac9-3661-4053-81cc-deaead1ad409-kube-api-access-68mrz\") pod \"maas-controller-9f656d559-798gc\" (UID: \"4615aac9-3661-4053-81cc-deaead1ad409\") " pod="opendatahub/maas-controller-9f656d559-798gc"
Sep 07 04:17:27.124237 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:27.124119    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9f656d559-798gc"
Sep 07 04:17:27.397633 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:27.397608    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9f656d559-798gc"]
Sep 07 04:17:27.399622 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:17:27.399599    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4615aac9_3661_4053_81cc_deaead1ad409.slice/crio-b2d87b13a3189839a027af2fb426b375af619ebdfa4d718a67755e3b28707356 WatchSource:0}: Error finding container b2d87b13a3189839a027af2fb426b375af619ebdfa4d718a67755e3b28707356: Status 404 returned error can't find the container with id b2d87b13a3189839a027af2fb426b375af619ebdfa4d718a67755e3b28707356
Sep 07 04:17:27.401064 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:27.401044    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 04:17:28.237401 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:28.237361    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9f656d559-798gc" event={"ID":"4615aac9-3661-4053-81cc-deaead1ad409","Type":"ContainerStarted","Data":"705185baad060329142d82d24df6f8db0a786ce772d010cd9c0d64176a3809fc"}
Sep 07 04:17:28.237827 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:28.237461    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-9f656d559-798gc"
Sep 07 04:17:28.237827 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:28.237486    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9f656d559-798gc" event={"ID":"4615aac9-3661-4053-81cc-deaead1ad409","Type":"ContainerStarted","Data":"b2d87b13a3189839a027af2fb426b375af619ebdfa4d718a67755e3b28707356"}
Sep 07 04:17:28.255158 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:28.255083    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-9f656d559-798gc" podStartSLOduration=1.64338373 podStartE2EDuration="2.255065382s" totalImagesPullingTime="611.681652ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:17:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:17:28.253083429 +0000 UTC m=+1735.186564148" watchObservedRunningTime="2026-09-07 04:17:28.255065382 +0000 UTC m=+1735.188546099"
Sep 07 04:17:39.248969 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:39.248932    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-9f656d559-798gc"
Sep 07 04:17:46.405176 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:46.405131    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:17:48.508809 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:48.508713    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:17:48.607892 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:48.607859    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-9f656d559-798gc"]
Sep 07 04:17:48.608104 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:48.608058    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-9f656d559-798gc" podUID="4615aac9-3661-4053-81cc-deaead1ad409" containerName="manager" containerID="cri-o://705185baad060329142d82d24df6f8db0a786ce772d010cd9c0d64176a3809fc" gracePeriod=30
Sep 07 04:17:48.856003 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:48.855978    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9f656d559-798gc"
Sep 07 04:17:48.895077 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:48.895044    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4615aac9-3661-4053-81cc-deaead1ad409-metrics-certs\") pod \"4615aac9-3661-4053-81cc-deaead1ad409\" (UID: \"4615aac9-3661-4053-81cc-deaead1ad409\") "
Sep 07 04:17:48.895274 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:48.895094    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68mrz\" (UniqueName: \"kubernetes.io/projected/4615aac9-3661-4053-81cc-deaead1ad409-kube-api-access-68mrz\") pod \"4615aac9-3661-4053-81cc-deaead1ad409\" (UID: \"4615aac9-3661-4053-81cc-deaead1ad409\") "
Sep 07 04:17:48.895274 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:48.895127    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4615aac9-3661-4053-81cc-deaead1ad409-webhook-cert\") pod \"4615aac9-3661-4053-81cc-deaead1ad409\" (UID: \"4615aac9-3661-4053-81cc-deaead1ad409\") "
Sep 07 04:17:48.897495 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:48.897448    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4615aac9-3661-4053-81cc-deaead1ad409-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "4615aac9-3661-4053-81cc-deaead1ad409" (UID: "4615aac9-3661-4053-81cc-deaead1ad409"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:17:48.897863 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:48.897841    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4615aac9-3661-4053-81cc-deaead1ad409-kube-api-access-68mrz" (OuterVolumeSpecName: "kube-api-access-68mrz") pod "4615aac9-3661-4053-81cc-deaead1ad409" (UID: "4615aac9-3661-4053-81cc-deaead1ad409"). InnerVolumeSpecName "kube-api-access-68mrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:17:48.897946 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:48.897841    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4615aac9-3661-4053-81cc-deaead1ad409-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4615aac9-3661-4053-81cc-deaead1ad409" (UID: "4615aac9-3661-4053-81cc-deaead1ad409"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:17:48.996185 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:48.996146    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4615aac9-3661-4053-81cc-deaead1ad409-metrics-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:17:48.996185 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:48.996180    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68mrz\" (UniqueName: \"kubernetes.io/projected/4615aac9-3661-4053-81cc-deaead1ad409-kube-api-access-68mrz\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:17:48.996185 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:48.996190    2583 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4615aac9-3661-4053-81cc-deaead1ad409-webhook-cert\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:17:49.346199 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:49.346167    2583 generic.go:356] "Generic (PLEG): container finished" podID="4615aac9-3661-4053-81cc-deaead1ad409" containerID="705185baad060329142d82d24df6f8db0a786ce772d010cd9c0d64176a3809fc" exitCode=0
Sep 07 04:17:49.346393 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:49.346246    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9f656d559-798gc" event={"ID":"4615aac9-3661-4053-81cc-deaead1ad409","Type":"ContainerDied","Data":"705185baad060329142d82d24df6f8db0a786ce772d010cd9c0d64176a3809fc"}
Sep 07 04:17:49.346393 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:49.346272    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9f656d559-798gc"
Sep 07 04:17:49.346393 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:49.346289    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9f656d559-798gc" event={"ID":"4615aac9-3661-4053-81cc-deaead1ad409","Type":"ContainerDied","Data":"b2d87b13a3189839a027af2fb426b375af619ebdfa4d718a67755e3b28707356"}
Sep 07 04:17:49.346393 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:49.346317    2583 scope.go:117] "RemoveContainer" containerID="705185baad060329142d82d24df6f8db0a786ce772d010cd9c0d64176a3809fc"
Sep 07 04:17:49.355865 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:49.355846    2583 scope.go:117] "RemoveContainer" containerID="705185baad060329142d82d24df6f8db0a786ce772d010cd9c0d64176a3809fc"
Sep 07 04:17:49.356181 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:17:49.356159    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"705185baad060329142d82d24df6f8db0a786ce772d010cd9c0d64176a3809fc\": container with ID starting with 705185baad060329142d82d24df6f8db0a786ce772d010cd9c0d64176a3809fc not found: ID does not exist" containerID="705185baad060329142d82d24df6f8db0a786ce772d010cd9c0d64176a3809fc"
Sep 07 04:17:49.356255 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:49.356195    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"705185baad060329142d82d24df6f8db0a786ce772d010cd9c0d64176a3809fc"} err="failed to get container status \"705185baad060329142d82d24df6f8db0a786ce772d010cd9c0d64176a3809fc\": rpc error: code = NotFound desc = could not find container \"705185baad060329142d82d24df6f8db0a786ce772d010cd9c0d64176a3809fc\": container with ID starting with 705185baad060329142d82d24df6f8db0a786ce772d010cd9c0d64176a3809fc not found: ID does not exist"
Sep 07 04:17:49.366375 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:49.366352    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-9f656d559-798gc"]
Sep 07 04:17:49.370195 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:49.370169    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-9f656d559-798gc"]
Sep 07 04:17:49.521878 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:49.521847    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4615aac9-3661-4053-81cc-deaead1ad409" path="/var/lib/kubelet/pods/4615aac9-3661-4053-81cc-deaead1ad409/volumes"
Sep 07 04:17:50.025550 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:50.025506    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:17:50.133467 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:50.133439    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-9f656d559-dljjg"]
Sep 07 04:17:50.133845 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:50.133832    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4615aac9-3661-4053-81cc-deaead1ad409" containerName="manager"
Sep 07 04:17:50.133886 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:50.133847    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4615aac9-3661-4053-81cc-deaead1ad409" containerName="manager"
Sep 07 04:17:50.133921 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:50.133901    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4615aac9-3661-4053-81cc-deaead1ad409" containerName="manager"
Sep 07 04:17:50.137149 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:50.137130    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9f656d559-dljjg"
Sep 07 04:17:50.140363 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:50.140330    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pl7hr\""
Sep 07 04:17:50.140505 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:50.140369    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 07 04:17:50.140505 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:50.140379    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 07 04:17:50.140505 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:50.140401    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 07 04:17:50.146531 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:50.146508    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9f656d559-dljjg"]
Sep 07 04:17:50.206106 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:50.206066    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb3274e2-eb23-4c18-ac39-e2ed2a6dc21e-webhook-cert\") pod \"maas-controller-9f656d559-dljjg\" (UID: \"eb3274e2-eb23-4c18-ac39-e2ed2a6dc21e\") " pod="opendatahub/maas-controller-9f656d559-dljjg"
Sep 07 04:17:50.206315 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:50.206113    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kgn4\" (UniqueName: \"kubernetes.io/projected/eb3274e2-eb23-4c18-ac39-e2ed2a6dc21e-kube-api-access-9kgn4\") pod \"maas-controller-9f656d559-dljjg\" (UID: \"eb3274e2-eb23-4c18-ac39-e2ed2a6dc21e\") " pod="opendatahub/maas-controller-9f656d559-dljjg"
Sep 07 04:17:50.206315 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:50.206134    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb3274e2-eb23-4c18-ac39-e2ed2a6dc21e-metrics-certs\") pod \"maas-controller-9f656d559-dljjg\" (UID: \"eb3274e2-eb23-4c18-ac39-e2ed2a6dc21e\") " pod="opendatahub/maas-controller-9f656d559-dljjg"
Sep 07 04:17:50.306707 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:50.306606    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb3274e2-eb23-4c18-ac39-e2ed2a6dc21e-webhook-cert\") pod \"maas-controller-9f656d559-dljjg\" (UID: \"eb3274e2-eb23-4c18-ac39-e2ed2a6dc21e\") " pod="opendatahub/maas-controller-9f656d559-dljjg"
Sep 07 04:17:50.306707 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:50.306650    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kgn4\" (UniqueName: \"kubernetes.io/projected/eb3274e2-eb23-4c18-ac39-e2ed2a6dc21e-kube-api-access-9kgn4\") pod \"maas-controller-9f656d559-dljjg\" (UID: \"eb3274e2-eb23-4c18-ac39-e2ed2a6dc21e\") " pod="opendatahub/maas-controller-9f656d559-dljjg"
Sep 07 04:17:50.306707 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:50.306669    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb3274e2-eb23-4c18-ac39-e2ed2a6dc21e-metrics-certs\") pod \"maas-controller-9f656d559-dljjg\" (UID: \"eb3274e2-eb23-4c18-ac39-e2ed2a6dc21e\") " pod="opendatahub/maas-controller-9f656d559-dljjg"
Sep 07 04:17:50.309289 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:50.309260    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb3274e2-eb23-4c18-ac39-e2ed2a6dc21e-metrics-certs\") pod \"maas-controller-9f656d559-dljjg\" (UID: \"eb3274e2-eb23-4c18-ac39-e2ed2a6dc21e\") " pod="opendatahub/maas-controller-9f656d559-dljjg"
Sep 07 04:17:50.309419 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:50.309373    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb3274e2-eb23-4c18-ac39-e2ed2a6dc21e-webhook-cert\") pod \"maas-controller-9f656d559-dljjg\" (UID: \"eb3274e2-eb23-4c18-ac39-e2ed2a6dc21e\") " pod="opendatahub/maas-controller-9f656d559-dljjg"
Sep 07 04:17:50.314755 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:50.314729    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kgn4\" (UniqueName: \"kubernetes.io/projected/eb3274e2-eb23-4c18-ac39-e2ed2a6dc21e-kube-api-access-9kgn4\") pod \"maas-controller-9f656d559-dljjg\" (UID: \"eb3274e2-eb23-4c18-ac39-e2ed2a6dc21e\") " pod="opendatahub/maas-controller-9f656d559-dljjg"
Sep 07 04:17:50.447829 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:50.447793    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9f656d559-dljjg"
Sep 07 04:17:50.717625 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:50.717598    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9f656d559-dljjg"]
Sep 07 04:17:50.720201 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:17:50.720162    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb3274e2_eb23_4c18_ac39_e2ed2a6dc21e.slice/crio-df51577906b374b2bfe47cbaa80b85930bea40c0cbe70a91f1d1ab79771585a8 WatchSource:0}: Error finding container df51577906b374b2bfe47cbaa80b85930bea40c0cbe70a91f1d1ab79771585a8: Status 404 returned error can't find the container with id df51577906b374b2bfe47cbaa80b85930bea40c0cbe70a91f1d1ab79771585a8
Sep 07 04:17:51.359205 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:51.359170    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9f656d559-dljjg" event={"ID":"eb3274e2-eb23-4c18-ac39-e2ed2a6dc21e","Type":"ContainerStarted","Data":"eb65229abae819b14f5ca1f11b7193d78704005bffbe11f1a20693191f5c5aa7"}
Sep 07 04:17:51.359404 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:51.359237    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9f656d559-dljjg" event={"ID":"eb3274e2-eb23-4c18-ac39-e2ed2a6dc21e","Type":"ContainerStarted","Data":"df51577906b374b2bfe47cbaa80b85930bea40c0cbe70a91f1d1ab79771585a8"}
Sep 07 04:17:51.359404 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:51.359350    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-9f656d559-dljjg"
Sep 07 04:17:51.375303 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:17:51.375153    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-9f656d559-dljjg" podStartSLOduration=0.937494935 podStartE2EDuration="1.375134275s" totalImagesPullingTime="437.63934ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:17:51.374602899 +0000 UTC m=+1758.308083616" watchObservedRunningTime="2026-09-07 04:17:51.375134275 +0000 UTC m=+1758.308615182"
Sep 07 04:18:02.366665 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:18:02.366635    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-9f656d559-dljjg"
Sep 07 04:18:09.691201 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:18:09.691155    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:18:09.745547 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:18:09.745516    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:19:13.841192 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:19:13.841156    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:20:35.159324 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:20:35.159288    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:20:46.595053 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:20:46.595012    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:21:17.381309 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.381275    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"]
Sep 07 04:21:17.385074 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.385049    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:17.387756 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.387727    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-9444e724-kserve-self-signed-certs\""
Sep 07 04:21:17.396620 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.396564    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"]
Sep 07 04:21:17.424188 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.424155    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:21:17.480563 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.480512    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-dshm\") pod \"e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") " pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:17.480563 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.480559    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-tmp-dir\") pod \"e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") " pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:17.480785 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.480594    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-home\") pod \"e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") " pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:17.480785 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.480661    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e164d288-892e-4c76-a97c-3da4d6760818-tls-certs\") pod \"e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") " pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:17.480785 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.480746    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gw4t\" (UniqueName: \"kubernetes.io/projected/e164d288-892e-4c76-a97c-3da4d6760818-kube-api-access-5gw4t\") pod \"e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") " pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:17.480951 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.480783    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-model-cache\") pod \"e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") " pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:17.581567 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.581519    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gw4t\" (UniqueName: \"kubernetes.io/projected/e164d288-892e-4c76-a97c-3da4d6760818-kube-api-access-5gw4t\") pod \"e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") " pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:17.581780 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.581576    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-model-cache\") pod \"e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") " pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:17.581780 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.581627    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-dshm\") pod \"e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") " pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:17.581780 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.581651    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-tmp-dir\") pod \"e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") " pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:17.581780 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.581678    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-home\") pod \"e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") " pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:17.581780 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.581731    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e164d288-892e-4c76-a97c-3da4d6760818-tls-certs\") pod \"e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") " pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:17.582060 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.582017    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-model-cache\") pod \"e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") " pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:17.582060 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.582041    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-home\") pod \"e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") " pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:17.582171 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.582119    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-tmp-dir\") pod \"e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") " pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:17.584228 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.584187    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-dshm\") pod \"e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") " pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:17.584449 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.584434    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e164d288-892e-4c76-a97c-3da4d6760818-tls-certs\") pod \"e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") " pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:17.590295 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.590268    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gw4t\" (UniqueName: \"kubernetes.io/projected/e164d288-892e-4c76-a97c-3da4d6760818-kube-api-access-5gw4t\") pod \"e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") " pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:17.698728 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.698686    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:17.979343 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:17.979161    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"]
Sep 07 04:21:17.981901 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:21:17.981878    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode164d288_892e_4c76_a97c_3da4d6760818.slice/crio-034f888d5d003a6c1acc4686cf4ee0c4a99e3c79e0eeb4ce5db8740a351a66bd WatchSource:0}: Error finding container 034f888d5d003a6c1acc4686cf4ee0c4a99e3c79e0eeb4ce5db8740a351a66bd: Status 404 returned error can't find the container with id 034f888d5d003a6c1acc4686cf4ee0c4a99e3c79e0eeb4ce5db8740a351a66bd
Sep 07 04:21:18.377885 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:18.377785    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn" event={"ID":"e164d288-892e-4c76-a97c-3da4d6760818","Type":"ContainerStarted","Data":"90ae557a39d61d22bca2e236e61e2336dfccb5cc06f0912b4b5414b2a65c3952"}
Sep 07 04:21:18.377885 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:18.377837    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn" event={"ID":"e164d288-892e-4c76-a97c-3da4d6760818","Type":"ContainerStarted","Data":"034f888d5d003a6c1acc4686cf4ee0c4a99e3c79e0eeb4ce5db8740a351a66bd"}
Sep 07 04:21:18.393871 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:18.393806    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn" podStartSLOduration=1.393789911 podStartE2EDuration="1.393789911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:21:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:21:18.392119965 +0000 UTC m=+1965.325600685" watchObservedRunningTime="2026-09-07 04:21:18.393789911 +0000 UTC m=+1965.327270629"
Sep 07 04:21:27.699490 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:27.699379    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:27.699490 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:27.699448    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:27.712425 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:27.712396    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:28.439082 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:28.439050    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:21:31.555044 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:31.555004    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:21:31.605390 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:31.605351    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:21:33.776008 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:21:33.775975    2583 scope.go:117] "RemoveContainer" containerID="577f54c14cb406211783f2c408f85bf2b4755d32d8afee3cee4a4d04810c3516"
Sep 07 04:22:10.144755 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:10.144719    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:22:11.383846 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:11.383807    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"]
Sep 07 04:22:11.397307 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:11.397266    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:22:26.405422 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.405345    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn" podUID="e164d288-892e-4c76-a97c-3da4d6760818" containerName="main" containerID="cri-o://90ae557a39d61d22bca2e236e61e2336dfccb5cc06f0912b4b5414b2a65c3952" gracePeriod=45
Sep 07 04:22:26.676152 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.676121    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:22:26.709424 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.709378    2583 generic.go:356] "Generic (PLEG): container finished" podID="e164d288-892e-4c76-a97c-3da4d6760818" containerID="90ae557a39d61d22bca2e236e61e2336dfccb5cc06f0912b4b5414b2a65c3952" exitCode=0
Sep 07 04:22:26.709627 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.709448    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn" event={"ID":"e164d288-892e-4c76-a97c-3da4d6760818","Type":"ContainerDied","Data":"90ae557a39d61d22bca2e236e61e2336dfccb5cc06f0912b4b5414b2a65c3952"}
Sep 07 04:22:26.709627 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.709488    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn" event={"ID":"e164d288-892e-4c76-a97c-3da4d6760818","Type":"ContainerDied","Data":"034f888d5d003a6c1acc4686cf4ee0c4a99e3c79e0eeb4ce5db8740a351a66bd"}
Sep 07 04:22:26.709627 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.709496    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"
Sep 07 04:22:26.709627 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.709509    2583 scope.go:117] "RemoveContainer" containerID="90ae557a39d61d22bca2e236e61e2336dfccb5cc06f0912b4b5414b2a65c3952"
Sep 07 04:22:26.718733 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.718711    2583 scope.go:117] "RemoveContainer" containerID="90ae557a39d61d22bca2e236e61e2336dfccb5cc06f0912b4b5414b2a65c3952"
Sep 07 04:22:26.719109 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:22:26.719086    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90ae557a39d61d22bca2e236e61e2336dfccb5cc06f0912b4b5414b2a65c3952\": container with ID starting with 90ae557a39d61d22bca2e236e61e2336dfccb5cc06f0912b4b5414b2a65c3952 not found: ID does not exist" containerID="90ae557a39d61d22bca2e236e61e2336dfccb5cc06f0912b4b5414b2a65c3952"
Sep 07 04:22:26.719203 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.719125    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90ae557a39d61d22bca2e236e61e2336dfccb5cc06f0912b4b5414b2a65c3952"} err="failed to get container status \"90ae557a39d61d22bca2e236e61e2336dfccb5cc06f0912b4b5414b2a65c3952\": rpc error: code = NotFound desc = could not find container \"90ae557a39d61d22bca2e236e61e2336dfccb5cc06f0912b4b5414b2a65c3952\": container with ID starting with 90ae557a39d61d22bca2e236e61e2336dfccb5cc06f0912b4b5414b2a65c3952 not found: ID does not exist"
Sep 07 04:22:26.836340 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.836298    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-dshm\") pod \"e164d288-892e-4c76-a97c-3da4d6760818\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") "
Sep 07 04:22:26.836534 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.836355    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-model-cache\") pod \"e164d288-892e-4c76-a97c-3da4d6760818\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") "
Sep 07 04:22:26.836534 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.836402    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-home\") pod \"e164d288-892e-4c76-a97c-3da4d6760818\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") "
Sep 07 04:22:26.836534 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.836429    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gw4t\" (UniqueName: \"kubernetes.io/projected/e164d288-892e-4c76-a97c-3da4d6760818-kube-api-access-5gw4t\") pod \"e164d288-892e-4c76-a97c-3da4d6760818\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") "
Sep 07 04:22:26.836534 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.836482    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e164d288-892e-4c76-a97c-3da4d6760818-tls-certs\") pod \"e164d288-892e-4c76-a97c-3da4d6760818\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") "
Sep 07 04:22:26.836534 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.836505    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-tmp-dir\") pod \"e164d288-892e-4c76-a97c-3da4d6760818\" (UID: \"e164d288-892e-4c76-a97c-3da4d6760818\") "
Sep 07 04:22:26.836801 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.836661    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-home" (OuterVolumeSpecName: "home") pod "e164d288-892e-4c76-a97c-3da4d6760818" (UID: "e164d288-892e-4c76-a97c-3da4d6760818"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:22:26.836801 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.836682    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-model-cache" (OuterVolumeSpecName: "model-cache") pod "e164d288-892e-4c76-a97c-3da4d6760818" (UID: "e164d288-892e-4c76-a97c-3da4d6760818"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:22:26.836894 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.836822    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e164d288-892e-4c76-a97c-3da4d6760818" (UID: "e164d288-892e-4c76-a97c-3da4d6760818"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:22:26.836957 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.836904    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-model-cache\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:22:26.836957 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.836922    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-home\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:22:26.838803 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.838770    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e164d288-892e-4c76-a97c-3da4d6760818-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e164d288-892e-4c76-a97c-3da4d6760818" (UID: "e164d288-892e-4c76-a97c-3da4d6760818"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:22:26.838803 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.838788    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-dshm" (OuterVolumeSpecName: "dshm") pod "e164d288-892e-4c76-a97c-3da4d6760818" (UID: "e164d288-892e-4c76-a97c-3da4d6760818"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 04:22:26.838971 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.838827    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e164d288-892e-4c76-a97c-3da4d6760818-kube-api-access-5gw4t" (OuterVolumeSpecName: "kube-api-access-5gw4t") pod "e164d288-892e-4c76-a97c-3da4d6760818" (UID: "e164d288-892e-4c76-a97c-3da4d6760818"). InnerVolumeSpecName "kube-api-access-5gw4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:22:26.938425 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.938392    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gw4t\" (UniqueName: \"kubernetes.io/projected/e164d288-892e-4c76-a97c-3da4d6760818-kube-api-access-5gw4t\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:22:26.938425 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.938424    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e164d288-892e-4c76-a97c-3da4d6760818-tls-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:22:26.938616 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.938455    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-tmp-dir\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:22:26.938616 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:26.938463    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e164d288-892e-4c76-a97c-3da4d6760818-dshm\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:22:27.031290 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:27.031258    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"]
Sep 07 04:22:27.035000 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:27.034967    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-9444e724-kserve-57bf8dc955-2fqnn"]
Sep 07 04:22:27.522288 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:27.522252    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e164d288-892e-4c76-a97c-3da4d6760818" path="/var/lib/kubelet/pods/e164d288-892e-4c76-a97c-3da4d6760818/volumes"
Sep 07 04:22:29.310514 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:29.310479    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-577644cfc5-ssv2w"]
Sep 07 04:22:29.311118 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:29.311098    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e164d288-892e-4c76-a97c-3da4d6760818" containerName="main"
Sep 07 04:22:29.311175 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:29.311123    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e164d288-892e-4c76-a97c-3da4d6760818" containerName="main"
Sep 07 04:22:29.311249 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:29.311238    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e164d288-892e-4c76-a97c-3da4d6760818" containerName="main"
Sep 07 04:22:29.315983 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:29.315964    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-577644cfc5-ssv2w"
Sep 07 04:22:29.325988 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:29.325958    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-577644cfc5-ssv2w"]
Sep 07 04:22:29.467428 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:29.467385    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fc0c844-32df-40d9-b638-7569316c70b9-openshift-service-ca\") pod \"maas-api-577644cfc5-ssv2w\" (UID: \"1fc0c844-32df-40d9-b638-7569316c70b9\") " pod="odh-ai-gateway-infra/maas-api-577644cfc5-ssv2w"
Sep 07 04:22:29.467599 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:29.467543    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fc0c844-32df-40d9-b638-7569316c70b9-maas-api-tls\") pod \"maas-api-577644cfc5-ssv2w\" (UID: \"1fc0c844-32df-40d9-b638-7569316c70b9\") " pod="odh-ai-gateway-infra/maas-api-577644cfc5-ssv2w"
Sep 07 04:22:29.467599 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:29.467572    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ccpv\" (UniqueName: \"kubernetes.io/projected/1fc0c844-32df-40d9-b638-7569316c70b9-kube-api-access-5ccpv\") pod \"maas-api-577644cfc5-ssv2w\" (UID: \"1fc0c844-32df-40d9-b638-7569316c70b9\") " pod="odh-ai-gateway-infra/maas-api-577644cfc5-ssv2w"
Sep 07 04:22:29.467599 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:29.467590    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1fc0c844-32df-40d9-b638-7569316c70b9-metrics-certs\") pod \"maas-api-577644cfc5-ssv2w\" (UID: \"1fc0c844-32df-40d9-b638-7569316c70b9\") " pod="odh-ai-gateway-infra/maas-api-577644cfc5-ssv2w"
Sep 07 04:22:29.568966 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:29.568877    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fc0c844-32df-40d9-b638-7569316c70b9-maas-api-tls\") pod \"maas-api-577644cfc5-ssv2w\" (UID: \"1fc0c844-32df-40d9-b638-7569316c70b9\") " pod="odh-ai-gateway-infra/maas-api-577644cfc5-ssv2w"
Sep 07 04:22:29.568966 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:29.568920    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ccpv\" (UniqueName: \"kubernetes.io/projected/1fc0c844-32df-40d9-b638-7569316c70b9-kube-api-access-5ccpv\") pod \"maas-api-577644cfc5-ssv2w\" (UID: \"1fc0c844-32df-40d9-b638-7569316c70b9\") " pod="odh-ai-gateway-infra/maas-api-577644cfc5-ssv2w"
Sep 07 04:22:29.568966 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:29.568938    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1fc0c844-32df-40d9-b638-7569316c70b9-metrics-certs\") pod \"maas-api-577644cfc5-ssv2w\" (UID: \"1fc0c844-32df-40d9-b638-7569316c70b9\") " pod="odh-ai-gateway-infra/maas-api-577644cfc5-ssv2w"
Sep 07 04:22:29.569200 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:29.568970    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fc0c844-32df-40d9-b638-7569316c70b9-openshift-service-ca\") pod \"maas-api-577644cfc5-ssv2w\" (UID: \"1fc0c844-32df-40d9-b638-7569316c70b9\") " pod="odh-ai-gateway-infra/maas-api-577644cfc5-ssv2w"
Sep 07 04:22:29.569788 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:29.569672    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fc0c844-32df-40d9-b638-7569316c70b9-openshift-service-ca\") pod \"maas-api-577644cfc5-ssv2w\" (UID: \"1fc0c844-32df-40d9-b638-7569316c70b9\") " pod="odh-ai-gateway-infra/maas-api-577644cfc5-ssv2w"
Sep 07 04:22:29.571624 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:29.571589    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fc0c844-32df-40d9-b638-7569316c70b9-maas-api-tls\") pod \"maas-api-577644cfc5-ssv2w\" (UID: \"1fc0c844-32df-40d9-b638-7569316c70b9\") " pod="odh-ai-gateway-infra/maas-api-577644cfc5-ssv2w"
Sep 07 04:22:29.571752 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:29.571713    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1fc0c844-32df-40d9-b638-7569316c70b9-metrics-certs\") pod \"maas-api-577644cfc5-ssv2w\" (UID: \"1fc0c844-32df-40d9-b638-7569316c70b9\") " pod="odh-ai-gateway-infra/maas-api-577644cfc5-ssv2w"
Sep 07 04:22:29.576627 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:29.576601    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ccpv\" (UniqueName: \"kubernetes.io/projected/1fc0c844-32df-40d9-b638-7569316c70b9-kube-api-access-5ccpv\") pod \"maas-api-577644cfc5-ssv2w\" (UID: \"1fc0c844-32df-40d9-b638-7569316c70b9\") " pod="odh-ai-gateway-infra/maas-api-577644cfc5-ssv2w"
Sep 07 04:22:29.629511 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:29.629473    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-577644cfc5-ssv2w"
Sep 07 04:22:29.907419 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:29.907390    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-577644cfc5-ssv2w"]
Sep 07 04:22:29.910674 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:22:29.910643    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fc0c844_32df_40d9_b638_7569316c70b9.slice/crio-e69cf4bd0a997accb251b5a02c2ef950f5f64df1fb3a06726fef6206441fdae2 WatchSource:0}: Error finding container e69cf4bd0a997accb251b5a02c2ef950f5f64df1fb3a06726fef6206441fdae2: Status 404 returned error can't find the container with id e69cf4bd0a997accb251b5a02c2ef950f5f64df1fb3a06726fef6206441fdae2
Sep 07 04:22:29.912043 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:29.912026    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 04:22:30.735320 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:30.735263    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-577644cfc5-ssv2w" event={"ID":"1fc0c844-32df-40d9-b638-7569316c70b9","Type":"ContainerStarted","Data":"84bb3b61200ac721ef2e1c7fa5510c0e4964885695ca371a9648e7ed358ba78d"}
Sep 07 04:22:30.735815 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:30.735348    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-577644cfc5-ssv2w"
Sep 07 04:22:30.735815 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:30.735379    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-577644cfc5-ssv2w" event={"ID":"1fc0c844-32df-40d9-b638-7569316c70b9","Type":"ContainerStarted","Data":"e69cf4bd0a997accb251b5a02c2ef950f5f64df1fb3a06726fef6206441fdae2"}
Sep 07 04:22:30.750958 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:30.750900    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-577644cfc5-ssv2w" podStartSLOduration=1.3185030420000001 podStartE2EDuration="1.750880895s" totalImagesPullingTime="432.377853ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 04:22:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:22:30.749948909 +0000 UTC m=+2037.683429628" watchObservedRunningTime="2026-09-07 04:22:30.750880895 +0000 UTC m=+2037.684361614"
Sep 07 04:22:36.747390 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:36.747357    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-577644cfc5-ssv2w"
Sep 07 04:22:36.807386 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:36.807351    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"]
Sep 07 04:22:36.807606 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:36.807576    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv" podUID="9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2" containerName="maas-api" containerID="cri-o://3bb3cdbacbb13a89327626aa50ffce1d4d2f87524a1406c497f7392419e7486a" gracePeriod=30
Sep 07 04:22:37.875136 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:37.875082    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv" podUID="9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.29:8443/health\": dial tcp 10.134.0.29:8443: connect: connection refused"
Sep 07 04:22:38.060841 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:38.060816    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"
Sep 07 04:22:38.150633 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:38.150543    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kp7qx\" (UniqueName: \"kubernetes.io/projected/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-kube-api-access-kp7qx\") pod \"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2\" (UID: \"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2\") "
Sep 07 04:22:38.150633 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:38.150587    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-openshift-service-ca\") pod \"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2\" (UID: \"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2\") "
Sep 07 04:22:38.150633 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:38.150618    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-maas-api-tls\") pod \"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2\" (UID: \"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2\") "
Sep 07 04:22:38.150949 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:38.150730    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-metrics-certs\") pod \"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2\" (UID: \"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2\") "
Sep 07 04:22:38.151011 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:38.150967    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2" (UID: "9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 04:22:38.151090 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:38.151065    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-openshift-service-ca\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:22:38.153014 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:38.152983    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2" (UID: "9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:22:38.153130 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:38.153023    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-kube-api-access-kp7qx" (OuterVolumeSpecName: "kube-api-access-kp7qx") pod "9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2" (UID: "9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2"). InnerVolumeSpecName "kube-api-access-kp7qx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 04:22:38.153130 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:38.153065    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2" (UID: "9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 04:22:38.252394 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:38.252355    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-metrics-certs\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:22:38.252394 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:38.252387    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kp7qx\" (UniqueName: \"kubernetes.io/projected/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-kube-api-access-kp7qx\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:22:38.252394 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:38.252397    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2-maas-api-tls\") on node \"ip-10-0-132-175.ec2.internal\" DevicePath \"\""
Sep 07 04:22:38.775317 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:38.775273    2583 generic.go:356] "Generic (PLEG): container finished" podID="9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2" containerID="3bb3cdbacbb13a89327626aa50ffce1d4d2f87524a1406c497f7392419e7486a" exitCode=0
Sep 07 04:22:38.775491 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:38.775327    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"
Sep 07 04:22:38.775491 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:38.775325    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv" event={"ID":"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2","Type":"ContainerDied","Data":"3bb3cdbacbb13a89327626aa50ffce1d4d2f87524a1406c497f7392419e7486a"}
Sep 07 04:22:38.775491 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:38.775446    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv" event={"ID":"9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2","Type":"ContainerDied","Data":"415009680b3d405143d193d8ea6994d2332512448390175eee1ebda32cb7abc9"}
Sep 07 04:22:38.775491 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:38.775467    2583 scope.go:117] "RemoveContainer" containerID="3bb3cdbacbb13a89327626aa50ffce1d4d2f87524a1406c497f7392419e7486a"
Sep 07 04:22:38.792122 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:38.792101    2583 scope.go:117] "RemoveContainer" containerID="3bb3cdbacbb13a89327626aa50ffce1d4d2f87524a1406c497f7392419e7486a"
Sep 07 04:22:38.792480 ip-10-0-132-175 kubenswrapper[2583]: E0907 04:22:38.792460    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bb3cdbacbb13a89327626aa50ffce1d4d2f87524a1406c497f7392419e7486a\": container with ID starting with 3bb3cdbacbb13a89327626aa50ffce1d4d2f87524a1406c497f7392419e7486a not found: ID does not exist" containerID="3bb3cdbacbb13a89327626aa50ffce1d4d2f87524a1406c497f7392419e7486a"
Sep 07 04:22:38.792530 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:38.792489    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bb3cdbacbb13a89327626aa50ffce1d4d2f87524a1406c497f7392419e7486a"} err="failed to get container status \"3bb3cdbacbb13a89327626aa50ffce1d4d2f87524a1406c497f7392419e7486a\": rpc error: code = NotFound desc = could not find container \"3bb3cdbacbb13a89327626aa50ffce1d4d2f87524a1406c497f7392419e7486a\": container with ID starting with 3bb3cdbacbb13a89327626aa50ffce1d4d2f87524a1406c497f7392419e7486a not found: ID does not exist"
Sep 07 04:22:38.799763 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:38.799731    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"]
Sep 07 04:22:38.806166 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:38.806135    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-86d6f5487f-wgcpv"]
Sep 07 04:22:39.522981 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:22:39.522943    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2" path="/var/lib/kubelet/pods/9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2/volumes"
Sep 07 04:23:08.350376 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:08.350277    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-69hxm_ef077a47-6929-4638-8d5a-7efdd731a387/manager/0.log"
Sep 07 04:23:08.463565 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:08.463530    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-9f656d559-dljjg_eb3274e2-eb23-4c18-ac39-e2ed2a6dc21e/manager/0.log"
Sep 07 04:23:08.808674 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:08.808641    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-756ff4ddd-m2jnd_863f75b6-d5da-405e-a695-be1c0eb08446/copy-manifests/0.log"
Sep 07 04:23:08.814965 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:08.814940    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-756ff4ddd-m2jnd_863f75b6-d5da-405e-a695-be1c0eb08446/manager/0.log"
Sep 07 04:23:09.516925 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:09.516898    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-577644cfc5-ssv2w_1fc0c844-32df-40d9-b638-7569316c70b9/maas-api/0.log"
Sep 07 04:23:09.834259 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:09.834145    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-ztxnh_252adbfb-596a-4262-beb3-f761ecf8aa0c/postgres/0.log"
Sep 07 04:23:10.813724 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:10.813696    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-2kdgv_03893c8b-3664-45ec-9df3-056fd805cdcb/manager/0.log"
Sep 07 04:23:10.922400 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:10.922365    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-n7x9r_05742da4-d5c2-4b32-b7ca-e2e0e66dfb60/kuadrant-console-plugin/0.log"
Sep 07 04:23:11.861452 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:11.861425    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59d7d88d64-m5zd6_f19fce4a-64b9-4988-b9bc-08f42b5c8843/kube-auth-proxy/0.log"
Sep 07 04:23:12.214106 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:12.214062    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-5m597_9ffd0e95-1557-4337-b585-7cd619dbc579/payload-pre-processing/0.log"
Sep 07 04:23:12.926148 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:12.926124    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-9td7c_dcf567d8-c91d-4855-8494-77316fc195be/main/0.log"
Sep 07 04:23:13.368853 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:13.368819    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-cpsc4_b52a8bd4-3572-41fc-95ed-cd4d884023b6/main/0.log"
Sep 07 04:23:13.475737 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:13.475704    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-9dgbf_05856c4d-524e-4235-b0aa-a5e0ecac3c23/main/0.log"
Sep 07 04:23:18.692699 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:18.692662    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:23:18.790696 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:18.790661    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:23:21.697145 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:21.697103    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-v9fzl_244e7a50-68b4-41dc-aa4b-8b83813cf489/global-pull-secret-syncer/0.log"
Sep 07 04:23:21.746350 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:21.746314    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log"
Sep 07 04:23:21.852422 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:21.852392    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-175.ec2.internal_0a39a5eb3a918e1ff8bcbe08348a2d2c/haproxy/0.log"
Sep 07 04:23:26.522756 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:26.522650    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-2kdgv_03893c8b-3664-45ec-9df3-056fd805cdcb/manager/0.log"
Sep 07 04:23:26.552410 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:26.552382    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-n7x9r_05742da4-d5c2-4b32-b7ca-e2e0e66dfb60/kuadrant-console-plugin/0.log"
Sep 07 04:23:28.396202 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:28.396103    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-t6fnv_a0500a2e-c89d-4b11-b01f-050faaa07bc5/kube-state-metrics/0.log"
Sep 07 04:23:28.411987 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:28.411961    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-t6fnv_a0500a2e-c89d-4b11-b01f-050faaa07bc5/kube-rbac-proxy-main/0.log"
Sep 07 04:23:28.435540 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:28.435514    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-t6fnv_a0500a2e-c89d-4b11-b01f-050faaa07bc5/kube-rbac-proxy-self/0.log"
Sep 07 04:23:28.513895 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:28.513862    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8r464_211cb420-6847-48a1-8067-7a0ef7238716/node-exporter/0.log"
Sep 07 04:23:28.541742 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:28.541715    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8r464_211cb420-6847-48a1-8067-7a0ef7238716/kube-rbac-proxy/0.log"
Sep 07 04:23:28.563121 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:28.563096    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8r464_211cb420-6847-48a1-8067-7a0ef7238716/init-textfile/0.log"
Sep 07 04:23:28.768753 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:28.768724    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-rwl8n_73a4703b-3eaf-4d46-a94f-c45676a063f3/kube-rbac-proxy-main/0.log"
Sep 07 04:23:28.798888 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:28.798858    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-rwl8n_73a4703b-3eaf-4d46-a94f-c45676a063f3/kube-rbac-proxy-self/0.log"
Sep 07 04:23:28.826749 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:28.826723    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-rwl8n_73a4703b-3eaf-4d46-a94f-c45676a063f3/openshift-state-metrics/0.log"
Sep 07 04:23:28.874448 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:28.874419    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6718f4c7-fcaa-4398-b866-dd03be6f1a57/prometheus/0.log"
Sep 07 04:23:28.903615 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:28.903589    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6718f4c7-fcaa-4398-b866-dd03be6f1a57/config-reloader/0.log"
Sep 07 04:23:28.932575 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:28.932555    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6718f4c7-fcaa-4398-b866-dd03be6f1a57/thanos-sidecar/0.log"
Sep 07 04:23:28.957769 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:28.957735    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6718f4c7-fcaa-4398-b866-dd03be6f1a57/kube-rbac-proxy-web/0.log"
Sep 07 04:23:28.980557 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:28.980528    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6718f4c7-fcaa-4398-b866-dd03be6f1a57/kube-rbac-proxy/0.log"
Sep 07 04:23:29.001898 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:29.001870    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6718f4c7-fcaa-4398-b866-dd03be6f1a57/kube-rbac-proxy-thanos/0.log"
Sep 07 04:23:29.029155 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:29.029072    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6718f4c7-fcaa-4398-b866-dd03be6f1a57/init-config-reloader/0.log"
Sep 07 04:23:29.057814 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:29.057785    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-5gzmw_af095918-3620-4785-b968-f2121d20d1db/prometheus-operator/0.log"
Sep 07 04:23:29.075339 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:29.075307    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-5gzmw_af095918-3620-4785-b968-f2121d20d1db/kube-rbac-proxy/0.log"
Sep 07 04:23:29.098276 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:29.098241    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-rdgxm_fe79a6a4-0529-4ff4-aabd-7c0a152f2878/prometheus-operator-admission-webhook/0.log"
Sep 07 04:23:29.217946 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:29.217916    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5765d7cf8b-pjfrx_0372d650-50ef-4bc7-a9e7-dd092bb32204/thanos-query/0.log"
Sep 07 04:23:29.240533 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:29.240512    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5765d7cf8b-pjfrx_0372d650-50ef-4bc7-a9e7-dd092bb32204/kube-rbac-proxy-web/0.log"
Sep 07 04:23:29.263201 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:29.263174    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5765d7cf8b-pjfrx_0372d650-50ef-4bc7-a9e7-dd092bb32204/kube-rbac-proxy/0.log"
Sep 07 04:23:29.284430 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:29.284360    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5765d7cf8b-pjfrx_0372d650-50ef-4bc7-a9e7-dd092bb32204/prom-label-proxy/0.log"
Sep 07 04:23:29.303573 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:29.303548    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5765d7cf8b-pjfrx_0372d650-50ef-4bc7-a9e7-dd092bb32204/kube-rbac-proxy-rules/0.log"
Sep 07 04:23:29.322380 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:29.322347    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5765d7cf8b-pjfrx_0372d650-50ef-4bc7-a9e7-dd092bb32204/kube-rbac-proxy-metrics/0.log"
Sep 07 04:23:31.521863 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.521831    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8"]
Sep 07 04:23:31.522265 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.522164    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2" containerName="maas-api"
Sep 07 04:23:31.522265 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.522176    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2" containerName="maas-api"
Sep 07 04:23:31.522345 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.522272    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e8cb9d1-b0ac-4f9e-82c3-06b790c2e0f2" containerName="maas-api"
Sep 07 04:23:31.525642 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.525618    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8"
Sep 07 04:23:31.528233 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.528182    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nrrkw\"/\"default-dockercfg-tlzxz\""
Sep 07 04:23:31.528385 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.528256    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nrrkw\"/\"openshift-service-ca.crt\""
Sep 07 04:23:31.528385 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.528270    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nrrkw\"/\"kube-root-ca.crt\""
Sep 07 04:23:31.529860 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.529836    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8"]
Sep 07 04:23:31.633312 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.633275    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knxqb\" (UniqueName: \"kubernetes.io/projected/be2f5b99-a512-4d2f-8838-5a0631efbc12-kube-api-access-knxqb\") pod \"perf-node-gather-daemonset-bdpj8\" (UID: \"be2f5b99-a512-4d2f-8838-5a0631efbc12\") " pod="openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8"
Sep 07 04:23:31.633523 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.633343    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be2f5b99-a512-4d2f-8838-5a0631efbc12-sys\") pod \"perf-node-gather-daemonset-bdpj8\" (UID: \"be2f5b99-a512-4d2f-8838-5a0631efbc12\") " pod="openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8"
Sep 07 04:23:31.633523 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.633375    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/be2f5b99-a512-4d2f-8838-5a0631efbc12-proc\") pod \"perf-node-gather-daemonset-bdpj8\" (UID: \"be2f5b99-a512-4d2f-8838-5a0631efbc12\") " pod="openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8"
Sep 07 04:23:31.633523 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.633504    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/be2f5b99-a512-4d2f-8838-5a0631efbc12-lib-modules\") pod \"perf-node-gather-daemonset-bdpj8\" (UID: \"be2f5b99-a512-4d2f-8838-5a0631efbc12\") " pod="openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8"
Sep 07 04:23:31.633689 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.633534    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/be2f5b99-a512-4d2f-8838-5a0631efbc12-podres\") pod \"perf-node-gather-daemonset-bdpj8\" (UID: \"be2f5b99-a512-4d2f-8838-5a0631efbc12\") " pod="openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8"
Sep 07 04:23:31.734677 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.734630    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/be2f5b99-a512-4d2f-8838-5a0631efbc12-lib-modules\") pod \"perf-node-gather-daemonset-bdpj8\" (UID: \"be2f5b99-a512-4d2f-8838-5a0631efbc12\") " pod="openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8"
Sep 07 04:23:31.734677 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.734672    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/be2f5b99-a512-4d2f-8838-5a0631efbc12-podres\") pod \"perf-node-gather-daemonset-bdpj8\" (UID: \"be2f5b99-a512-4d2f-8838-5a0631efbc12\") " pod="openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8"
Sep 07 04:23:31.734941 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.734701    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knxqb\" (UniqueName: \"kubernetes.io/projected/be2f5b99-a512-4d2f-8838-5a0631efbc12-kube-api-access-knxqb\") pod \"perf-node-gather-daemonset-bdpj8\" (UID: \"be2f5b99-a512-4d2f-8838-5a0631efbc12\") " pod="openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8"
Sep 07 04:23:31.734941 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.734729    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be2f5b99-a512-4d2f-8838-5a0631efbc12-sys\") pod \"perf-node-gather-daemonset-bdpj8\" (UID: \"be2f5b99-a512-4d2f-8838-5a0631efbc12\") " pod="openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8"
Sep 07 04:23:31.734941 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.734749    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/be2f5b99-a512-4d2f-8838-5a0631efbc12-proc\") pod \"perf-node-gather-daemonset-bdpj8\" (UID: \"be2f5b99-a512-4d2f-8838-5a0631efbc12\") " pod="openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8"
Sep 07 04:23:31.734941 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.734839    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/be2f5b99-a512-4d2f-8838-5a0631efbc12-podres\") pod \"perf-node-gather-daemonset-bdpj8\" (UID: \"be2f5b99-a512-4d2f-8838-5a0631efbc12\") " pod="openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8"
Sep 07 04:23:31.734941 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.734847    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/be2f5b99-a512-4d2f-8838-5a0631efbc12-proc\") pod \"perf-node-gather-daemonset-bdpj8\" (UID: \"be2f5b99-a512-4d2f-8838-5a0631efbc12\") " pod="openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8"
Sep 07 04:23:31.734941 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.734837    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/be2f5b99-a512-4d2f-8838-5a0631efbc12-lib-modules\") pod \"perf-node-gather-daemonset-bdpj8\" (UID: \"be2f5b99-a512-4d2f-8838-5a0631efbc12\") " pod="openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8"
Sep 07 04:23:31.734941 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.734879    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be2f5b99-a512-4d2f-8838-5a0631efbc12-sys\") pod \"perf-node-gather-daemonset-bdpj8\" (UID: \"be2f5b99-a512-4d2f-8838-5a0631efbc12\") " pod="openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8"
Sep 07 04:23:31.744979 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.744950    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knxqb\" (UniqueName: \"kubernetes.io/projected/be2f5b99-a512-4d2f-8838-5a0631efbc12-kube-api-access-knxqb\") pod \"perf-node-gather-daemonset-bdpj8\" (UID: \"be2f5b99-a512-4d2f-8838-5a0631efbc12\") " pod="openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8"
Sep 07 04:23:31.837359 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.837269    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8"
Sep 07 04:23:31.897281 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:31.897194    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-lkmpv_8d993695-6e46-4b29-98da-6ba5256072a3/download-server/0.log"
Sep 07 04:23:32.112006 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:32.111978    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8"]
Sep 07 04:23:32.114868 ip-10-0-132-175 kubenswrapper[2583]: W0907 04:23:32.114831    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbe2f5b99_a512_4d2f_8838_5a0631efbc12.slice/crio-e25b7334f25614dd14005d90da1eb4a06c1b440e6f1d214c84834b9fd2bb85c4 WatchSource:0}: Error finding container e25b7334f25614dd14005d90da1eb4a06c1b440e6f1d214c84834b9fd2bb85c4: Status 404 returned error can't find the container with id e25b7334f25614dd14005d90da1eb4a06c1b440e6f1d214c84834b9fd2bb85c4
Sep 07 04:23:33.043638 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:33.043602    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8" event={"ID":"be2f5b99-a512-4d2f-8838-5a0631efbc12","Type":"ContainerStarted","Data":"cae34cbe4246b11989a3af3f36ee11d1c6f0b1f9d80682ed96fd92eea5ed914e"}
Sep 07 04:23:33.044059 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:33.043653    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8" event={"ID":"be2f5b99-a512-4d2f-8838-5a0631efbc12","Type":"ContainerStarted","Data":"e25b7334f25614dd14005d90da1eb4a06c1b440e6f1d214c84834b9fd2bb85c4"}
Sep 07 04:23:33.044059 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:33.043706    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8"
Sep 07 04:23:33.058568 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:33.058517    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8" podStartSLOduration=2.058501136 podStartE2EDuration="2.058501136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 04:23:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 04:23:33.057266841 +0000 UTC m=+2099.990747560" watchObservedRunningTime="2026-09-07 04:23:33.058501136 +0000 UTC m=+2099.991981884"
Sep 07 04:23:33.400500 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:33.400420    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gw9hg_f5e84b44-6d3b-464b-a987-fff4a928f773/konnectivity-agent/0.log" limit=1024
Sep 07 04:23:33.667053 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:33.666975    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gcl7p_f35afb94-a0dc-4039-8d88-81f6fc8326c0/dns/0.log"
Sep 07 04:23:33.686121 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:33.686091    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gcl7p_f35afb94-a0dc-4039-8d88-81f6fc8326c0/kube-rbac-proxy/0.log"
Sep 07 04:23:33.822923 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:33.822904    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-px55t_503f1518-10bb-424b-87b0-00bf7cf2239f/dns-node-resolver/0.log"
Sep 07 04:23:34.404693 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:34.404663    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xg5w5_52683f85-6093-473d-9e11-541e6ec8b2fb/node-ca/0.log"
Sep 07 04:23:35.899864 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:35.899813    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59d7d88d64-m5zd6_f19fce4a-64b9-4988-b9bc-08f42b5c8843/kube-auth-proxy/0.log"
Sep 07 04:23:35.984025 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:35.983994    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-5m597_9ffd0e95-1557-4337-b585-7cd619dbc579/payload-pre-processing/0.log"
Sep 07 04:23:36.929187 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:36.929145    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-k7hdq_822aab36-17d2-448a-a74c-273fe4450e29/serve-healthcheck-canary/0.log"
Sep 07 04:23:37.475941 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:37.475915    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nmxn8_1a092764-0c63-4c0b-8456-a05adb098222/kube-rbac-proxy/0.log"
Sep 07 04:23:37.495375 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:37.495347    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nmxn8_1a092764-0c63-4c0b-8456-a05adb098222/exporter/0.log"
Sep 07 04:23:37.515495 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:37.515461    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nmxn8_1a092764-0c63-4c0b-8456-a05adb098222/extractor/0.log"
Sep 07 04:23:39.059492 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:39.059465    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nrrkw/perf-node-gather-daemonset-bdpj8"
Sep 07 04:23:39.932952 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:39.932924    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-69hxm_ef077a47-6929-4638-8d5a-7efdd731a387/manager/0.log"
Sep 07 04:23:39.980396 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:39.980362    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-9f656d559-dljjg_eb3274e2-eb23-4c18-ac39-e2ed2a6dc21e/manager/0.log"
Sep 07 04:23:40.160657 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:40.160626    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-756ff4ddd-m2jnd_863f75b6-d5da-405e-a695-be1c0eb08446/manager/0.log"
Sep 07 04:23:40.185540 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:40.185519    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-756ff4ddd-m2jnd_863f75b6-d5da-405e-a695-be1c0eb08446/copy-manifests/0.log"
Sep 07 04:23:41.811770 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:41.811696    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-758b7d97c6-dp9p2_0da7680f-9fb0-46ea-a483-7085afac2772/manager/0.log"
Sep 07 04:23:42.787510 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:42.787476    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59d7d88d64-m5zd6_f19fce4a-64b9-4988-b9bc-08f42b5c8843/kube-auth-proxy/0.log"
Sep 07 04:23:42.860791 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:42.860760    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-5m597_9ffd0e95-1557-4337-b585-7cd619dbc579/payload-pre-processing/0.log"
Sep 07 04:23:44.927276 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:44.927249    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59d7d88d64-m5zd6_f19fce4a-64b9-4988-b9bc-08f42b5c8843/kube-auth-proxy/0.log"
Sep 07 04:23:44.997152 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:44.997121    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-5m597_9ffd0e95-1557-4337-b585-7cd619dbc579/payload-pre-processing/0.log"
Sep 07 04:23:48.173573 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:48.173540    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9cztj_8c1667f2-8f99-44ed-96de-544fea7fe5a6/kube-multus/0.log"
Sep 07 04:23:48.397136 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:48.397106    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-prd5g_de17335a-7920-400b-ad6d-73fe656c84e0/kube-multus-additional-cni-plugins/0.log"
Sep 07 04:23:48.417749 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:48.417717    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-prd5g_de17335a-7920-400b-ad6d-73fe656c84e0/egress-router-binary-copy/0.log"
Sep 07 04:23:48.435287 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:48.435260    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-prd5g_de17335a-7920-400b-ad6d-73fe656c84e0/cni-plugins/0.log"
Sep 07 04:23:48.452361 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:48.452336    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-prd5g_de17335a-7920-400b-ad6d-73fe656c84e0/bond-cni-plugin/0.log"
Sep 07 04:23:48.471167 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:48.471143    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-prd5g_de17335a-7920-400b-ad6d-73fe656c84e0/routeoverride-cni/0.log"
Sep 07 04:23:48.492350 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:48.492321    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-prd5g_de17335a-7920-400b-ad6d-73fe656c84e0/whereabouts-cni-bincopy/0.log"
Sep 07 04:23:48.513054 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:48.513030    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-prd5g_de17335a-7920-400b-ad6d-73fe656c84e0/whereabouts-cni/0.log"
Sep 07 04:23:48.759580 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:48.759504    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-sml8b_bb13bb24-ca38-4d13-aa0c-1c47eb028ae1/network-metrics-daemon/0.log"
Sep 07 04:23:48.778610 ip-10-0-132-175 kubenswrapper[2583]: I0907 04:23:48.778582    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-sml8b_bb13bb24-ca38-4d13-aa0c-1c47eb028ae1/kube-rbac-proxy/0.log"