Jul 30 17:27:05.414115 ip-10-0-135-203 systemd[1]: Starting Kubernetes Kubelet...
Jul 30 17:27:05.819767 ip-10-0-135-203 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 17:27:05.819767 ip-10-0-135-203 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 30 17:27:05.819767 ip-10-0-135-203 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 17:27:05.819767 ip-10-0-135-203 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 30 17:27:05.819767 ip-10-0-135-203 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 17:27:05.820467 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.819765    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 30 17:27:05.825310 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825266    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 17:27:05.825310 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825307    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 17:27:05.825310 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825312    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 17:27:05.825310 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825316    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 17:27:05.825507 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825321    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 17:27:05.825507 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825325    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 17:27:05.825507 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825329    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 17:27:05.825507 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825333    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 17:27:05.825507 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825337    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 17:27:05.825507 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825342    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 17:27:05.825507 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825346    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 17:27:05.825507 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825350    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 17:27:05.825507 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825354    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 17:27:05.825507 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825359    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 17:27:05.825507 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825363    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 17:27:05.825507 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825367    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 17:27:05.825507 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825371    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 17:27:05.825507 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825374    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 17:27:05.825507 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825377    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 17:27:05.825507 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825380    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 17:27:05.825507 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825382    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 17:27:05.825507 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825385    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 17:27:05.825507 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825387    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 17:27:05.825507 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825392    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 17:27:05.826000 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825397    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 17:27:05.826000 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825400    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 17:27:05.826000 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825403    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 17:27:05.826000 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825406    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 17:27:05.826000 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825409    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 17:27:05.826000 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825412    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 17:27:05.826000 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825414    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 17:27:05.826000 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825416    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 17:27:05.826000 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825419    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 17:27:05.826000 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825422    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 17:27:05.826000 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825424    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 17:27:05.826000 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825427    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 17:27:05.826000 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825430    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 17:27:05.826000 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825433    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 17:27:05.826000 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825435    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 17:27:05.826000 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825437    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 17:27:05.826000 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825440    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 17:27:05.826000 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825442    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 17:27:05.826000 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825445    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 17:27:05.826000 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825447    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 17:27:05.826520 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825450    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 17:27:05.826520 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825452    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 17:27:05.826520 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825454    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 17:27:05.826520 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825457    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 17:27:05.826520 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825459    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 17:27:05.826520 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825462    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 17:27:05.826520 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825477    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 17:27:05.826520 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825479    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 17:27:05.826520 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825482    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 17:27:05.826520 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825485    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 17:27:05.826520 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825487    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 17:27:05.826520 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825490    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 17:27:05.826520 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825493    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 17:27:05.826520 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825496    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 17:27:05.826520 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825498    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 17:27:05.826520 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825501    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 17:27:05.826520 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825504    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 17:27:05.826520 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825507    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 17:27:05.826520 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825510    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 17:27:05.826995 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825513    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 17:27:05.826995 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825515    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 17:27:05.826995 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825518    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 17:27:05.826995 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825521    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 17:27:05.826995 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825524    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 17:27:05.826995 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825527    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 17:27:05.826995 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825530    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 17:27:05.826995 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825534    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 17:27:05.826995 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825538    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 17:27:05.826995 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825541    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 17:27:05.826995 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825543    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 17:27:05.826995 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825546    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 17:27:05.826995 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825549    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 17:27:05.826995 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825551    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 17:27:05.826995 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825553    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 17:27:05.826995 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825556    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 17:27:05.826995 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825560    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 17:27:05.826995 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825563    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 17:27:05.826995 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825565    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 17:27:05.826995 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825567    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 17:27:05.827489 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825576    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 17:27:05.827489 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825578    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 17:27:05.827489 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825581    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 17:27:05.827489 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825584    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 17:27:05.827489 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825586    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 17:27:05.827489 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825589    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 17:27:05.827489 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825592    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 17:27:05.827489 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825595    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 17:27:05.827489 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825597    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 17:27:05.827489 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825600    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 17:27:05.827489 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825603    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 17:27:05.827489 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825607    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 17:27:05.827489 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825610    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 17:27:05.827489 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825613    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 17:27:05.827489 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825615    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 17:27:05.827489 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.825618    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 17:27:05.827489 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826456    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 17:27:05.827489 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826465    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 17:27:05.827489 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826468    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 17:27:05.827489 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826471    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 17:27:05.827986 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826474    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 17:27:05.827986 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826478    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 17:27:05.827986 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826482    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 17:27:05.827986 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826485    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 17:27:05.827986 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826487    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 17:27:05.827986 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826490    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 17:27:05.827986 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826494    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 17:27:05.827986 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826497    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 17:27:05.827986 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826500    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 17:27:05.827986 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826502    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 17:27:05.827986 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826505    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 17:27:05.827986 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826507    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 17:27:05.827986 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826509    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 17:27:05.827986 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826512    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 17:27:05.827986 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826515    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 17:27:05.827986 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826517    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 17:27:05.827986 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826520    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 17:27:05.827986 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826522    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 17:27:05.827986 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826525    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 17:27:05.828527 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826528    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 17:27:05.828527 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826530    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 17:27:05.828527 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826532    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 17:27:05.828527 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826535    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 17:27:05.828527 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826538    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 17:27:05.828527 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826540    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 17:27:05.828527 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826542    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 17:27:05.828527 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826545    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 17:27:05.828527 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826547    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 17:27:05.828527 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826549    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 17:27:05.828527 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826552    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 17:27:05.828527 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826555    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 17:27:05.828527 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826557    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 17:27:05.828527 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826560    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 17:27:05.828527 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826562    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 17:27:05.828527 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826565    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 17:27:05.828527 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826567    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 17:27:05.828527 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826570    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 17:27:05.828527 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826572    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 17:27:05.828527 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826575    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 17:27:05.829024 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826577    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 17:27:05.829024 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826580    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 17:27:05.829024 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826582    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 17:27:05.829024 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826585    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 17:27:05.829024 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826587    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 17:27:05.829024 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826590    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 17:27:05.829024 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826592    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 17:27:05.829024 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826596    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 17:27:05.829024 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826598    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 17:27:05.829024 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826600    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 17:27:05.829024 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826603    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 17:27:05.829024 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826605    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 17:27:05.829024 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826607    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 17:27:05.829024 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826610    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 17:27:05.829024 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826612    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 17:27:05.829024 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826615    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 17:27:05.829024 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826617    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 17:27:05.829024 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826619    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 17:27:05.829024 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826621    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 17:27:05.829024 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826624    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 17:27:05.829524 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826626    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 17:27:05.829524 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826628    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 17:27:05.829524 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826631    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 17:27:05.829524 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826634    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 17:27:05.829524 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826637    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 17:27:05.829524 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826639    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 17:27:05.829524 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826641    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 17:27:05.829524 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826644    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 17:27:05.829524 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826646    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 17:27:05.829524 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826648    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 17:27:05.829524 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826651    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 17:27:05.829524 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826653    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 17:27:05.829524 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826655    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 17:27:05.829524 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826658    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 17:27:05.829524 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826660    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 17:27:05.829524 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826663    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 17:27:05.829524 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826666    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 17:27:05.829524 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826668    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 17:27:05.829524 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826671    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 17:27:05.829524 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826673    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 17:27:05.830023 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826676    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 17:27:05.830023 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826678    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 17:27:05.830023 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826682    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 17:27:05.830023 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826686    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 17:27:05.830023 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826688    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 17:27:05.830023 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826691    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 17:27:05.830023 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826694    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 17:27:05.830023 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826696    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 17:27:05.830023 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826699    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 17:27:05.830023 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826701    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 17:27:05.830023 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826704    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 17:27:05.830023 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826706    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 17:27:05.830023 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826709    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 17:27:05.830023 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826711    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 17:27:05.830023 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826714    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 17:27:05.830023 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.826716    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 17:27:05.830023 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827104    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 17:27:05.830023 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827110    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 17:27:05.830023 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827113    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 17:27:05.830512 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827116    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 17:27:05.830512 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827118    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 17:27:05.830512 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827121    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 17:27:05.830512 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827123    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 17:27:05.830512 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827126    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 17:27:05.830512 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827129    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 17:27:05.830512 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827131    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 17:27:05.830512 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827134    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 17:27:05.830512 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827136    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 17:27:05.830512 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827140    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 17:27:05.830512 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827142    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 17:27:05.830512 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827145    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 17:27:05.830512 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827147    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 17:27:05.830512 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827150    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 17:27:05.830512 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827153    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 17:27:05.830512 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827156    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 17:27:05.830512 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827159    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 17:27:05.830512 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827165    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 17:27:05.830512 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827169    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 17:27:05.830512 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827173    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 17:27:05.831009 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827178    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 17:27:05.831009 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827180    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 17:27:05.831009 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827183    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 17:27:05.831009 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827185    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 17:27:05.831009 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827188    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 17:27:05.831009 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827191    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 17:27:05.831009 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827193    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 17:27:05.831009 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827196    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 17:27:05.831009 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827198    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 17:27:05.831009 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827201    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 17:27:05.831009 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827204    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 17:27:05.831009 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827206    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 17:27:05.831009 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827208    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 17:27:05.831009 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827211    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 17:27:05.831009 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827213    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 17:27:05.831009 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827216    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 17:27:05.831009 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827218    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 17:27:05.831009 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827221    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 17:27:05.831009 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827224    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 17:27:05.831452 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827226    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 17:27:05.831452 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827228    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 17:27:05.831452 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827233    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 17:27:05.831452 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827236    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 17:27:05.831452 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827238    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 17:27:05.831452 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827241    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 17:27:05.831452 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827243    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 17:27:05.831452 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827246    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 17:27:05.831452 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827249    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 17:27:05.831452 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827251    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 17:27:05.831452 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827255    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 17:27:05.831452 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827258    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 17:27:05.831452 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827262    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 17:27:05.831452 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827265    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 17:27:05.831452 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827268    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 17:27:05.831452 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827270    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 17:27:05.831452 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827273    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 17:27:05.831452 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827275    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 17:27:05.831452 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827278    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 17:27:05.831452 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827280    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 17:27:05.831967 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827282    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 17:27:05.831967 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827285    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 17:27:05.831967 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827287    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 17:27:05.831967 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827289    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 17:27:05.831967 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827293    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 17:27:05.831967 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827295    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 17:27:05.831967 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827298    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 17:27:05.831967 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827300    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 17:27:05.831967 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827303    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 17:27:05.831967 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827305    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 17:27:05.831967 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827308    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 17:27:05.831967 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827310    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 17:27:05.831967 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827312    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 17:27:05.831967 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827315    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 17:27:05.831967 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827318    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 17:27:05.831967 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827321    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 17:27:05.831967 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827323    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 17:27:05.831967 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827326    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 17:27:05.831967 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827328    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 17:27:05.831967 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827330    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 17:27:05.832438 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827333    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 17:27:05.832438 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827336    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 17:27:05.832438 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827338    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 17:27:05.832438 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827341    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 17:27:05.832438 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827345    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 17:27:05.832438 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827347    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 17:27:05.832438 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827351    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 17:27:05.832438 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827354    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 17:27:05.832438 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827357    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 17:27:05.832438 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827359    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 17:27:05.832438 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827362    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 17:27:05.832438 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827364    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 17:27:05.832438 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827367    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 17:27:05.832438 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827369    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 17:27:05.832438 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827372    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 17:27:05.832438 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827374    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 17:27:05.832438 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.827376    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 17:27:05.832438 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827449    2577 flags.go:64] FLAG: --address="0.0.0.0"
Jul 30 17:27:05.832438 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827456    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 30 17:27:05.832438 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827462    2577 flags.go:64] FLAG: --anonymous-auth="true"
Jul 30 17:27:05.832933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827466    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 30 17:27:05.832933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827472    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 30 17:27:05.832933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827477    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 30 17:27:05.832933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827484    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 30 17:27:05.832933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827490    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 30 17:27:05.832933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827495    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 30 17:27:05.832933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827499    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 30 17:27:05.832933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827502    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 30 17:27:05.832933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827505    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 30 17:27:05.832933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827508    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 30 17:27:05.832933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827511    2577 flags.go:64] FLAG: --cgroup-root=""
Jul 30 17:27:05.832933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827513    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 30 17:27:05.832933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827516    2577 flags.go:64] FLAG: --client-ca-file=""
Jul 30 17:27:05.832933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827520    2577 flags.go:64] FLAG: --cloud-provider="external"
Jul 30 17:27:05.832933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827522    2577 flags.go:64] FLAG: --cluster-dns="[]"
Jul 30 17:27:05.832933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827526    2577 flags.go:64] FLAG: --cluster-domain=""
Jul 30 17:27:05.832933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827530    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 30 17:27:05.832933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827533    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 30 17:27:05.832933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827536    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 30 17:27:05.832933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827539    2577 flags.go:64] FLAG: --container-log-max-files="5"
Jul 30 17:27:05.832933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827543    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 30 17:27:05.832933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827545    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 30 17:27:05.832933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827548    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827551    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827554    2577 flags.go:64] FLAG: --contention-profiling="false"
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827557    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827559    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827562    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827565    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827569    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827572    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827575    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827578    2577 flags.go:64] FLAG: --enable-load-reader="false"
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827580    2577 flags.go:64] FLAG: --enable-server="true"
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827583    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827588    2577 flags.go:64] FLAG: --event-burst="100"
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827591    2577 flags.go:64] FLAG: --event-qps="50"
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827593    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827597    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827600    2577 flags.go:64] FLAG: --eviction-hard=""
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827604    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827606    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827609    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827612    2577 flags.go:64] FLAG: --eviction-soft=""
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827615    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827617    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827621    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 30 17:27:05.833521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827624    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827627    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827630    2577 flags.go:64] FLAG: --fail-swap-on="true"
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827633    2577 flags.go:64] FLAG: --feature-gates=""
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827636    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827639    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827642    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827645    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827647    2577 flags.go:64] FLAG: --healthz-port="10248"
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827650    2577 flags.go:64] FLAG: --help="false"
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827653    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-135-203.ec2.internal"
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827656    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827658    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827661    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827664    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827667    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827670    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827672    2577 flags.go:64] FLAG: --image-service-endpoint=""
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827675    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827678    2577 flags.go:64] FLAG: --kube-api-burst="100"
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827680    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827683    2577 flags.go:64] FLAG: --kube-api-qps="50"
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827686    2577 flags.go:64] FLAG: --kube-reserved=""
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827690    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 30 17:27:05.834159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827693    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827696    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827698    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827701    2577 flags.go:64] FLAG: --lock-file=""
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827703    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827706    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827709    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827714    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827717    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827721    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827724    2577 flags.go:64] FLAG: --logging-format="text"
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827727    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827730    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827733    2577 flags.go:64] FLAG: --manifest-url=""
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827735    2577 flags.go:64] FLAG: --manifest-url-header=""
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827740    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827742    2577 flags.go:64] FLAG: --max-open-files="1000000"
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827746    2577 flags.go:64] FLAG: --max-pods="110"
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827749    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827752    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827754    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827757    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827760    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827763    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827766    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 30 17:27:05.834754 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827772    2577 flags.go:64] FLAG: --node-status-max-images="50"
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827775    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827778    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827781    2577 flags.go:64] FLAG: --pod-cidr=""
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827784    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827788    2577 flags.go:64] FLAG: --pod-manifest-path=""
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827791    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827795    2577 flags.go:64] FLAG: --pods-per-core="0"
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827797    2577 flags.go:64] FLAG: --port="10250"
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827800    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827803    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08fb893af134baf35"
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827806    2577 flags.go:64] FLAG: --qos-reserved=""
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827809    2577 flags.go:64] FLAG: --read-only-port="10255"
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827828    2577 flags.go:64] FLAG: --register-node="true"
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827834    2577 flags.go:64] FLAG: --register-with-taints=""
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827840    2577 flags.go:64] FLAG: --registry-burst="10"
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827845    2577 flags.go:64] FLAG: --registry-qps="5"
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827848    2577 flags.go:64] FLAG: --reserved-cpus=""
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827851    2577 flags.go:64] FLAG: --reserved-memory=""
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827854    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827857    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827860    2577 flags.go:64] FLAG: --rotate-certificates="false"
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827862    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827865    2577 flags.go:64] FLAG: --runonce="false"
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827868    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 30 17:27:05.835367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827871    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827874    2577 flags.go:64] FLAG: --seccomp-default="false"
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827876    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827879    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827882    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827884    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827887    2577 flags.go:64] FLAG: --storage-driver-password="root"
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827890    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827893    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827895    2577 flags.go:64] FLAG: --storage-driver-user="root"
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827898    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827901    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827903    2577 flags.go:64] FLAG: --system-cgroups=""
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827906    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827912    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827915    2577 flags.go:64] FLAG: --tls-cert-file=""
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827917    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827921    2577 flags.go:64] FLAG: --tls-min-version=""
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827924    2577 flags.go:64] FLAG: --tls-private-key-file=""
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827927    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827930    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827932    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827935    2577 flags.go:64] FLAG: --v="2"
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827940    2577 flags.go:64] FLAG: --version="false"
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827948    2577 flags.go:64] FLAG: --vmodule=""
Jul 30 17:27:05.836034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827952    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 30 17:27:05.836627 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.827955    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 30 17:27:05.836627 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828066    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 17:27:05.836627 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828070    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 17:27:05.836627 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828073    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 17:27:05.836627 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828076    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 17:27:05.836627 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828079    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 17:27:05.836627 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828082    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 17:27:05.836627 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828084    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 17:27:05.836627 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828087    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 17:27:05.836627 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828090    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 17:27:05.836627 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828092    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 17:27:05.836627 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828095    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 17:27:05.836627 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828098    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 17:27:05.836627 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828100    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 17:27:05.836627 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828102    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 17:27:05.836627 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828105    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 17:27:05.836627 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828107    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 17:27:05.836627 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828110    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 17:27:05.836627 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828112    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 17:27:05.836627 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828115    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 17:27:05.837161 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828118    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 17:27:05.837161 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828120    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 17:27:05.837161 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828123    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 17:27:05.837161 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828125    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 17:27:05.837161 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828127    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 17:27:05.837161 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828130    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 17:27:05.837161 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828132    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 17:27:05.837161 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828135    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 17:27:05.837161 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828137    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 17:27:05.837161 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828141    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 17:27:05.837161 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828143    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 17:27:05.837161 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828146    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 17:27:05.837161 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828148    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 17:27:05.837161 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828152    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 17:27:05.837161 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828155    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 17:27:05.837161 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828158    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 17:27:05.837161 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828161    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 17:27:05.837161 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828163    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 17:27:05.837161 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828165    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 17:27:05.837161 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828168    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 17:27:05.837676 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828170    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 17:27:05.837676 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828172    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 17:27:05.837676 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828175    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 17:27:05.837676 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828177    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 17:27:05.837676 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828179    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 17:27:05.837676 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828182    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 17:27:05.837676 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828184    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 17:27:05.837676 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828186    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 17:27:05.837676 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828189    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 17:27:05.837676 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828191    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 17:27:05.837676 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828193    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 17:27:05.837676 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828196    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 17:27:05.837676 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828198    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 17:27:05.837676 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828201    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 17:27:05.837676 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828204    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 17:27:05.837676 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828206    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 17:27:05.837676 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828208    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 17:27:05.837676 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828211    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 17:27:05.837676 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828213    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 17:27:05.837676 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828215    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 17:27:05.838186 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828218    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 17:27:05.838186 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828221    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 17:27:05.838186 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828223    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 17:27:05.838186 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828226    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 17:27:05.838186 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828228    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 17:27:05.838186 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828231    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 17:27:05.838186 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828233    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 17:27:05.838186 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828235    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 17:27:05.838186 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828238    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 17:27:05.838186 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828240    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 17:27:05.838186 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828242    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 17:27:05.838186 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828245    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 17:27:05.838186 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828247    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 17:27:05.838186 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828249    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 17:27:05.838186 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828252    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 17:27:05.838186 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828254    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 17:27:05.838186 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828258    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 17:27:05.838186 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828261    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 17:27:05.838186 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828264    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 17:27:05.838663 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828266    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 17:27:05.838663 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828269    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 17:27:05.838663 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828272    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 17:27:05.838663 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828274    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 17:27:05.838663 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828277    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 17:27:05.838663 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828279    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 17:27:05.838663 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828283    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 17:27:05.838663 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828287    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 17:27:05.838663 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828289    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 17:27:05.838663 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828292    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 17:27:05.838663 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828294    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 17:27:05.838663 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828296    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 17:27:05.838663 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828299    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 17:27:05.838663 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828301    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 17:27:05.838663 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828305    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 17:27:05.838663 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828307    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 17:27:05.838663 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828310    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 17:27:05.838663 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828312    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 17:27:05.838663 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828319    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 17:27:05.839130 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828321    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 17:27:05.839130 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.828324    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 17:27:05.839130 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.829277    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 17:27:05.839130 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.836954    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 30 17:27:05.839130 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.836971    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 30 17:27:05.839130 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.837756    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 30 17:27:05.839130 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.837764    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 30 17:27:05.839130 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837855    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 17:27:05.839130 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837861    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 17:27:05.839130 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837864    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 17:27:05.839130 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837867    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 17:27:05.839130 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837870    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 17:27:05.839130 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837874    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 17:27:05.839130 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837876    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 17:27:05.839130 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837879    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 17:27:05.839494 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837881    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 17:27:05.839494 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837884    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 17:27:05.839494 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837886    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 17:27:05.839494 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837888    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 17:27:05.839494 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837891    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 17:27:05.839494 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837893    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 17:27:05.839494 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837896    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 17:27:05.839494 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837900    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 17:27:05.839494 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837903    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 17:27:05.839494 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837907    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 17:27:05.839494 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837909    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 17:27:05.839494 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837912    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 17:27:05.839494 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837915    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 17:27:05.839494 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837918    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 17:27:05.839494 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837920    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 17:27:05.839494 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837923    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 17:27:05.839494 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837926    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 17:27:05.839494 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837929    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 17:27:05.839494 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837931    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 17:27:05.839494 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837934    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 17:27:05.839979 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837937    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 17:27:05.839979 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837941    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 17:27:05.839979 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837943    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 17:27:05.839979 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837946    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 17:27:05.839979 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837949    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 17:27:05.839979 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837951    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 17:27:05.839979 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837954    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 17:27:05.839979 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837956    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 17:27:05.839979 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837959    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 17:27:05.839979 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837961    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 17:27:05.839979 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837963    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 17:27:05.839979 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837966    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 17:27:05.839979 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837969    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 17:27:05.839979 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837973    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 17:27:05.839979 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837976    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 17:27:05.839979 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837979    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 17:27:05.839979 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837982    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 17:27:05.839979 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837984    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 17:27:05.839979 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837987    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 17:27:05.840424 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837990    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 17:27:05.840424 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837992    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 17:27:05.840424 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837995    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 17:27:05.840424 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.837997    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 17:27:05.840424 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838000    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 17:27:05.840424 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838003    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 17:27:05.840424 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838006    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 17:27:05.840424 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838008    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 17:27:05.840424 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838011    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 17:27:05.840424 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838013    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 17:27:05.840424 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838015    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 17:27:05.840424 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838018    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 17:27:05.840424 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838020    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 17:27:05.840424 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838023    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 17:27:05.840424 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838026    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 17:27:05.840424 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838029    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 17:27:05.840424 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838032    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 17:27:05.840424 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838035    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 17:27:05.840424 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838038    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 17:27:05.840424 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838041    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 17:27:05.840919 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838044    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 17:27:05.840919 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838047    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 17:27:05.840919 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838049    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 17:27:05.840919 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838052    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 17:27:05.840919 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838054    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 17:27:05.840919 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838056    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 17:27:05.840919 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838059    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 17:27:05.840919 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838061    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 17:27:05.840919 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838064    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 17:27:05.840919 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838066    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 17:27:05.840919 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838069    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 17:27:05.840919 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838072    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 17:27:05.840919 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838074    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 17:27:05.840919 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838077    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 17:27:05.840919 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838079    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 17:27:05.840919 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838082    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 17:27:05.840919 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838084    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 17:27:05.840919 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838087    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 17:27:05.840919 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838090    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 17:27:05.841400 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838092    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 17:27:05.841400 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838094    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 17:27:05.841400 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838097    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 17:27:05.841400 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838100    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 17:27:05.841400 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838102    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 17:27:05.841400 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838105    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 17:27:05.841400 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838108    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 17:27:05.841400 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838111    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 17:27:05.841400 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838113    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 17:27:05.841400 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838116    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 17:27:05.841400 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838119    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 17:27:05.841400 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838122    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 17:27:05.841400 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838124    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 17:27:05.841400 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.838129    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 17:27:05.841400 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838248    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 17:27:05.841775 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838253    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 17:27:05.841775 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838256    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 17:27:05.841775 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838259    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 17:27:05.841775 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838262    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 17:27:05.841775 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838265    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 17:27:05.841775 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838268    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 17:27:05.841775 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838271    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 17:27:05.841775 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838273    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 17:27:05.841775 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838276    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 17:27:05.841775 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838278    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 17:27:05.841775 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838281    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 17:27:05.841775 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838283    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 17:27:05.841775 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838286    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 17:27:05.841775 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838289    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 17:27:05.841775 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838291    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 17:27:05.841775 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838294    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 17:27:05.841775 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838296    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 17:27:05.841775 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838299    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 17:27:05.841775 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838301    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 17:27:05.841775 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838304    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 17:27:05.842387 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838306    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 17:27:05.842387 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838309    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 17:27:05.842387 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838311    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 17:27:05.842387 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838314    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 17:27:05.842387 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838316    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 17:27:05.842387 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838320    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 17:27:05.842387 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838324    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 17:27:05.842387 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838327    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 17:27:05.842387 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838330    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 17:27:05.842387 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838333    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 17:27:05.842387 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838335    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 17:27:05.842387 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838338    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 17:27:05.842387 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838341    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 17:27:05.842387 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838343    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 17:27:05.842387 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838346    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 17:27:05.842387 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838348    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 17:27:05.842387 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838352    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 17:27:05.842387 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838355    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 17:27:05.842387 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838357    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 17:27:05.842898 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838360    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 17:27:05.842898 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838363    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 17:27:05.842898 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838365    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 17:27:05.842898 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838368    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 17:27:05.842898 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838370    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 17:27:05.842898 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838373    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 17:27:05.842898 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838375    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 17:27:05.842898 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838378    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 17:27:05.842898 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838380    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 17:27:05.842898 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838383    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 17:27:05.842898 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838385    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 17:27:05.842898 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838388    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 17:27:05.842898 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838390    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 17:27:05.842898 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838405    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 17:27:05.842898 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838408    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 17:27:05.842898 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838411    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 17:27:05.842898 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838413    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 17:27:05.842898 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838415    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 17:27:05.842898 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838418    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 17:27:05.842898 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838420    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 17:27:05.843376 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838423    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 17:27:05.843376 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838426    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 17:27:05.843376 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838429    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 17:27:05.843376 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838431    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 17:27:05.843376 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838433    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 17:27:05.843376 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838436    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 17:27:05.843376 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838438    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 17:27:05.843376 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838441    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 17:27:05.843376 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838443    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 17:27:05.843376 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838445    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 17:27:05.843376 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838447    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 17:27:05.843376 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838450    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 17:27:05.843376 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838452    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 17:27:05.843376 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838454    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 17:27:05.843376 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838457    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 17:27:05.843376 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838459    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 17:27:05.843376 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838461    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 17:27:05.843376 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838463    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 17:27:05.843376 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838466    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 17:27:05.843376 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838469    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 17:27:05.843918 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838472    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 17:27:05.843918 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838474    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 17:27:05.843918 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838476    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 17:27:05.843918 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838479    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 17:27:05.843918 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838481    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 17:27:05.843918 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838484    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 17:27:05.843918 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838486    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 17:27:05.843918 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838489    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 17:27:05.843918 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838491    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 17:27:05.843918 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838493    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 17:27:05.843918 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838495    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 17:27:05.843918 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838498    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 17:27:05.843918 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838500    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 17:27:05.843918 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838502    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 17:27:05.843918 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838505    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 17:27:05.843918 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838508    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 17:27:05.843918 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838510    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 17:27:05.843918 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838512    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 17:27:05.843918 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:05.838514    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 17:27:05.844359 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.838519    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 17:27:05.844359 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.838633    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 30 17:27:05.844359 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.841876    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 30 17:27:05.844359 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.843345    2577 server.go:1019] "Starting client certificate rotation"
Jul 30 17:27:05.844359 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.843466    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 17:27:05.844552 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.844408    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 17:27:05.846050 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.846035    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 17:27:05.849016 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.848996    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 17:27:05.859601 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.859575    2577 log.go:25] "Validated CRI v1 runtime API"
Jul 30 17:27:05.863517 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.863501    2577 log.go:25] "Validated CRI v1 image API"
Jul 30 17:27:05.864391 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.864375    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 30 17:27:05.867541 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.867523    2577 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ea79a776-5984-4a60-8566-e550277dd780:/dev/nvme0n1p3 fd9e3ec2-e67d-43fd-aa08-6735f330f3a0:/dev/nvme0n1p4]
Jul 30 17:27:05.867610 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.867542    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 30 17:27:05.872428 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.872322    2577 manager.go:216] Machine: {Timestamp:2026-07-30 17:27:05.871194854 +0000 UTC m=+0.324758835 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3143664 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2fefffcb92d5749cde0599810ebe9a SystemUUID:ec2fefff-cb92-d574-9cde-0599810ebe9a BootID:fc00d241-a357-488e-a0fe-fd87e63190e9 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:db:b1:44:01:cf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:db:b1:44:01:cf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ae:e7:2b:37:1b:02 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 30 17:27:05.872428 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.872426    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 30 17:27:05.872527 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.872502    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 30 17:27:05.875558 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.875518    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 30 17:27:05.875678 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.875662    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 17:27:05.875722 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.875566    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-203.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 30 17:27:05.876409 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.876399    2577 topology_manager.go:138] "Creating topology manager with none policy"
Jul 30 17:27:05.876445 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.876411    2577 container_manager_linux.go:309] "Creating device plugin manager"
Jul 30 17:27:05.876445 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.876419    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 17:27:05.877103 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.877091    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 17:27:05.878407 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.878395    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 30 17:27:05.880234 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.880223    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 17:27:05.880354 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.880344    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 30 17:27:05.880382 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.880374    2577 kubelet.go:486] "Attempting to sync node with API server"
Jul 30 17:27:05.880409 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.880383    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 30 17:27:05.880409 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.880394    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 30 17:27:05.880409 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.880405    2577 kubelet.go:398] "Adding apiserver pod source"
Jul 30 17:27:05.880495 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.880414    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 30 17:27:05.880869 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.880856    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 17:27:05.880910 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.880880    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 17:27:05.883934 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.883918    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 30 17:27:05.886357 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.886342    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 30 17:27:05.886449 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.886373    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 30 17:27:05.888234 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.888222    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 30 17:27:05.888281 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.888237    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 30 17:27:05.888281 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.888243    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 30 17:27:05.888281 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.888249    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 30 17:27:05.888281 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.888257    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 30 17:27:05.888281 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.888262    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 30 17:27:05.888281 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.888270    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 30 17:27:05.888281 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.888274    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 30 17:27:05.888480 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.888286    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 30 17:27:05.888480 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.888291    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 30 17:27:05.888480 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.888309    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 30 17:27:05.888480 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.888314    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 30 17:27:05.888906 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.888897    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 30 17:27:05.888964 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.888907    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 30 17:27:05.890747 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:05.890727    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 30 17:27:05.890826 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:05.890727    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-203.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 30 17:27:05.892068 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.892056    2577 server.go:1268] "Started kubelet"
Jul 30 17:27:05.892168 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.892122    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 30 17:27:05.892236 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.892204    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 30 17:27:05.892299 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.892258    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 30 17:27:05.896201 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.895931    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 30 17:27:05.896241 ip-10-0-135-203 systemd[1]: Started Kubernetes Kubelet.
Jul 30 17:27:05.898837 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.897426    2577 server.go:310] "Adding debug handlers to kubelet server"
Jul 30 17:27:05.899191 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:05.899172    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 30 17:27:05.904435 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.904416    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-203.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 30 17:27:05.905241 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:05.904446    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-203.ec2.internal.18c72154ebc21463  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-203.ec2.internal,UID:ip-10-0-135-203.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-203.ec2.internal,},FirstTimestamp:2026-07-30 17:27:05.892025443 +0000 UTC m=+0.345589424,LastTimestamp:2026-07-30 17:27:05.892025443 +0000 UTC m=+0.345589424,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-203.ec2.internal,}"
Jul 30 17:27:05.905675 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.905659    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 30 17:27:05.905675 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.905669    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 30 17:27:05.905771 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:05.905749    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-203.ec2.internal\" not found"
Jul 30 17:27:05.906213 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.906196    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 30 17:27:05.906213 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.906215    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 30 17:27:05.906357 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.906343    2577 reconstruct.go:97] "Volume reconstruction finished"
Jul 30 17:27:05.906357 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.906353    2577 reconciler.go:29] "Reconciler: start to sync state"
Jul 30 17:27:05.906476 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.906427    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 30 17:27:05.906625 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.906602    2577 factory.go:55] Registering systemd factory
Jul 30 17:27:05.906625 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.906625    2577 factory.go:223] Registration of the systemd container factory successfully
Jul 30 17:27:05.908102 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.908083    2577 factory.go:153] Registering CRI-O factory
Jul 30 17:27:05.908102 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.908103    2577 factory.go:223] Registration of the crio container factory successfully
Jul 30 17:27:05.908238 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.908169    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 30 17:27:05.908238 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.908192    2577 factory.go:103] Registering Raw factory
Jul 30 17:27:05.908238 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.908206    2577 manager.go:1196] Started watching for new ooms in manager
Jul 30 17:27:05.908609 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.908594    2577 manager.go:349] Starting recovery of all containers
Jul 30 17:27:05.909716 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:05.909687    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-203.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 30 17:27:05.909789 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:05.909768    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 30 17:27:05.911998 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.911976    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vvcvc"
Jul 30 17:27:05.917027 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.917010    2577 manager.go:354] Recovery completed
Jul 30 17:27:05.919145 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.919128    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vvcvc"
Jul 30 17:27:05.922544 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.922459    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 17:27:05.928525 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.928509    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-203.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 17:27:05.928609 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.928544    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-203.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 17:27:05.928609 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.928556    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-203.ec2.internal" event="NodeHasSufficientPID"
Jul 30 17:27:05.929118 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.929106    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 30 17:27:05.929206 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.929118    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 30 17:27:05.929206 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.929128    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 17:27:05.929206 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.929135    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 17:27:05.929909 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.929892    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-203.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 17:27:05.929996 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.929921    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-203.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 17:27:05.929996 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.929933    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-203.ec2.internal" event="NodeHasSufficientPID"
Jul 30 17:27:05.930566 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:05.930487    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-203.ec2.internal.18c72154edef06c2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-203.ec2.internal,UID:ip-10-0-135-203.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-203.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-203.ec2.internal,},FirstTimestamp:2026-07-30 17:27:05.928525506 +0000 UTC m=+0.382089489,LastTimestamp:2026-07-30 17:27:05.928525506 +0000 UTC m=+0.382089489,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-203.ec2.internal,}"
Jul 30 17:27:05.931229 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.931219    2577 policy_none.go:49] "None policy: Start"
Jul 30 17:27:05.931278 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.931235    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 30 17:27:05.931278 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.931249    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 30 17:27:05.932348 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.932337    2577 policy_none.go:47] "Start"
Jul 30 17:27:05.968397 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.968381    2577 manager.go:337] "Starting Device Plugin manager"
Jul 30 17:27:05.968522 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:05.968415    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 30 17:27:05.968522 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.968427    2577 server.go:87] "Starting device plugin registration server"
Jul 30 17:27:05.968746 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.968733    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 30 17:27:05.968791 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.968747    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 30 17:27:05.968898 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.968856    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 30 17:27:05.968962 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.968947    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 30 17:27:05.968962 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:05.968961    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 30 17:27:05.969519 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:05.969497    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 30 17:27:05.969616 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:05.969543    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-203.ec2.internal\" not found"
Jul 30 17:27:06.069443 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.069234    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 17:27:06.070603 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.070475    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-203.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 17:27:06.070603 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.070509    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-203.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 17:27:06.070603 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.070522    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-203.ec2.internal" event="NodeHasSufficientPID"
Jul 30 17:27:06.070603 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.070549    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-203.ec2.internal"
Jul 30 17:27:06.077611 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.077595    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-203.ec2.internal"
Jul 30 17:27:06.077672 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:06.077619    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-203.ec2.internal\": node \"ip-10-0-135-203.ec2.internal\" not found"
Jul 30 17:27:06.088005 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:06.087984    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-203.ec2.internal\" not found"
Jul 30 17:27:06.102478 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.102429    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 30 17:27:06.103851 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.103838    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 30 17:27:06.103904 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.103871    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 30 17:27:06.103904 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.103891    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 30 17:27:06.103977 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:06.103924    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 30 17:27:06.107123 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.107106    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 17:27:06.188981 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:06.188947    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-203.ec2.internal\" not found"
Jul 30 17:27:06.204121 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.204093    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-203.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-203.ec2.internal"]
Jul 30 17:27:06.204198 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.204188    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 17:27:06.205768 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.205751    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-203.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 17:27:06.205875 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.205782    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-203.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 17:27:06.205875 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.205799    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-203.ec2.internal" event="NodeHasSufficientPID"
Jul 30 17:27:06.207274 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.207262    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 17:27:06.207321 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.207288    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/64aa76e700834f6545b825fca32978ba-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-203.ec2.internal\" (UID: \"64aa76e700834f6545b825fca32978ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-203.ec2.internal"
Jul 30 17:27:06.207321 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.207313    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64aa76e700834f6545b825fca32978ba-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-203.ec2.internal\" (UID: \"64aa76e700834f6545b825fca32978ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-203.ec2.internal"
Jul 30 17:27:06.207422 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.207408    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-203.ec2.internal"
Jul 30 17:27:06.207451 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.207438    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 17:27:06.208016 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.207999    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-203.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 17:27:06.208016 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.208010    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-203.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 17:27:06.208158 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.208024    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-203.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 17:27:06.208158 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.208028    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-203.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 17:27:06.208158 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.208039    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-203.ec2.internal" event="NodeHasSufficientPID"
Jul 30 17:27:06.208158 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.208055    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-203.ec2.internal" event="NodeHasSufficientPID"
Jul 30 17:27:06.209361 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.209341    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-203.ec2.internal"
Jul 30 17:27:06.209361 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.209367    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 17:27:06.210051 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.210036    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-203.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 17:27:06.210124 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.210063    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-203.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 17:27:06.210124 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.210078    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-203.ec2.internal" event="NodeHasSufficientPID"
Jul 30 17:27:06.231428 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:06.231407    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-203.ec2.internal\" not found" node="ip-10-0-135-203.ec2.internal"
Jul 30 17:27:06.235864 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:06.235848    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-203.ec2.internal\" not found" node="ip-10-0-135-203.ec2.internal"
Jul 30 17:27:06.289687 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:06.289660    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-203.ec2.internal\" not found"
Jul 30 17:27:06.307435 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.307412    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/64aa76e700834f6545b825fca32978ba-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-203.ec2.internal\" (UID: \"64aa76e700834f6545b825fca32978ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-203.ec2.internal"
Jul 30 17:27:06.307495 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.307439    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64aa76e700834f6545b825fca32978ba-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-203.ec2.internal\" (UID: \"64aa76e700834f6545b825fca32978ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-203.ec2.internal"
Jul 30 17:27:06.307533 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.307498    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ebe8f1131fdea96935597faec1e85919-config\") pod \"kube-apiserver-proxy-ip-10-0-135-203.ec2.internal\" (UID: \"ebe8f1131fdea96935597faec1e85919\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-203.ec2.internal"
Jul 30 17:27:06.307533 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.307520    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/64aa76e700834f6545b825fca32978ba-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-203.ec2.internal\" (UID: \"64aa76e700834f6545b825fca32978ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-203.ec2.internal"
Jul 30 17:27:06.307591 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.307547    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64aa76e700834f6545b825fca32978ba-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-203.ec2.internal\" (UID: \"64aa76e700834f6545b825fca32978ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-203.ec2.internal"
Jul 30 17:27:06.390756 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:06.390702    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-203.ec2.internal\" not found"
Jul 30 17:27:06.408411 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.408389    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ebe8f1131fdea96935597faec1e85919-config\") pod \"kube-apiserver-proxy-ip-10-0-135-203.ec2.internal\" (UID: \"ebe8f1131fdea96935597faec1e85919\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-203.ec2.internal"
Jul 30 17:27:06.408463 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.408437    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ebe8f1131fdea96935597faec1e85919-config\") pod \"kube-apiserver-proxy-ip-10-0-135-203.ec2.internal\" (UID: \"ebe8f1131fdea96935597faec1e85919\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-203.ec2.internal"
Jul 30 17:27:06.491637 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:06.491593    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-203.ec2.internal\" not found"
Jul 30 17:27:06.535087 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.535059    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-203.ec2.internal"
Jul 30 17:27:06.539887 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.539696    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-203.ec2.internal"
Jul 30 17:27:06.592687 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:06.592659    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-203.ec2.internal\" not found"
Jul 30 17:27:06.693194 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:06.693129    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-203.ec2.internal\" not found"
Jul 30 17:27:06.793703 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:06.793672    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-203.ec2.internal\" not found"
Jul 30 17:27:06.843125 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.843104    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 30 17:27:06.843507 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.843255    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 17:27:06.893776 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:06.893756    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-203.ec2.internal\" not found"
Jul 30 17:27:06.893851 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.893784    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 17:27:06.925114 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.925076    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-29 17:22:06 +0000 UTC" deadline="2028-02-08 11:44:35.210385015 +0000 UTC"
Jul 30 17:27:06.925114 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:06.925111    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13386h17m28.285277386s"
Jul 30 17:27:06.994769 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:06.994692    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-203.ec2.internal\" not found"
Jul 30 17:27:07.095333 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:07.095284    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-203.ec2.internal\" not found"
Jul 30 17:27:07.170661 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.170632    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 17:27:07.206043 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.206015    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-203.ec2.internal"
Jul 30 17:27:07.228291 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.228266    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 17:27:07.229919 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.229899    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-203.ec2.internal"
Jul 30 17:27:07.248204 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.248184    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 17:27:07.258126 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:07.258095    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64aa76e700834f6545b825fca32978ba.slice/crio-eaedc5ae09cd858eda1b7c64f02cfdb158095207e9bdbac4d055d8f0f49a957c WatchSource:0}: Error finding container eaedc5ae09cd858eda1b7c64f02cfdb158095207e9bdbac4d055d8f0f49a957c: Status 404 returned error can't find the container with id eaedc5ae09cd858eda1b7c64f02cfdb158095207e9bdbac4d055d8f0f49a957c
Jul 30 17:27:07.258593 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:07.258572    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebe8f1131fdea96935597faec1e85919.slice/crio-9cd304e0e1c307810be38e774ed1a064bc4ff5c09469ef37a91d15b15faa191a WatchSource:0}: Error finding container 9cd304e0e1c307810be38e774ed1a064bc4ff5c09469ef37a91d15b15faa191a: Status 404 returned error can't find the container with id 9cd304e0e1c307810be38e774ed1a064bc4ff5c09469ef37a91d15b15faa191a
Jul 30 17:27:07.264909 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.264890    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 17:27:07.427771 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.427742    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 17:27:07.882100 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.882064    2577 apiserver.go:52] "Watching apiserver"
Jul 30 17:27:07.900648 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.900617    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 30 17:27:07.902127 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.902095    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-ctb9m","openshift-network-diagnostics/network-check-target-vm97k","openshift-network-operator/iptables-alerter-ppvt7","openshift-ovn-kubernetes/ovnkube-node-b6r8l","kube-system/global-pull-secret-syncer-f4vp7","kube-system/konnectivity-agent-6wttc","kube-system/kube-apiserver-proxy-ip-10-0-135-203.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm","openshift-dns/node-resolver-mxb44","openshift-multus/multus-9nsjh","openshift-multus/network-metrics-daemon-lwrkg","openshift-cluster-node-tuning-operator/tuned-sfrfh","openshift-image-registry/node-ca-hkwdn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-203.ec2.internal"]
Jul 30 17:27:07.904073 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.904049    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm"
Jul 30 17:27:07.905507 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.905489    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:07.905593 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:07.905573    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vm97k" podUID="98fcc6f5-705d-415c-b08c-87dc618baca2"
Jul 30 17:27:07.906874 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.906853    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ppvt7"
Jul 30 17:27:07.907654 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.907631    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 30 17:27:07.908571 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.908551    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:07.915269 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.915252    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6wttc"
Jul 30 17:27:07.915359 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.915346    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mxb44"
Jul 30 17:27:07.915612 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.915593    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:07.915691 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.915676    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:07.915744 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:07.915698    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwrkg" podUID="845cc472-f650-4adc-81b0-94b11e29ecfd"
Jul 30 17:27:07.918285 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.916785    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 30 17:27:07.918285 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.916996    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 30 17:27:07.918285 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.917143    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 30 17:27:07.918285 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.917343    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 30 17:27:07.918285 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.917495    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 30 17:27:07.918285 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.917510    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-25b2d\""
Jul 30 17:27:07.918285 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.917717    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 30 17:27:07.918285 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.917719    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 30 17:27:07.918285 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.917867    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:07.918285 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.918190    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:07.918285 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:07.918230    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4vp7" podUID="a3251e0d-bc32-44a8-848d-2e64df31b7cc"
Jul 30 17:27:07.918932 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.918797    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/667a3ad7-25b2-430a-9878-94f8e7fa9069-sys-fs\") pod \"aws-ebs-csi-driver-node-fk7lm\" (UID: \"667a3ad7-25b2-430a-9878-94f8e7fa9069\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm"
Jul 30 17:27:07.918932 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.918819    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnfv6\" (UniqueName: \"kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6\") pod \"network-check-target-vm97k\" (UID: \"98fcc6f5-705d-415c-b08c-87dc618baca2\") " pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:07.918932 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.918886    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-systemd-units\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:07.919098 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.918943    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-run-systemd\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:07.919098 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.918991    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/667a3ad7-25b2-430a-9878-94f8e7fa9069-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fk7lm\" (UID: \"667a3ad7-25b2-430a-9878-94f8e7fa9069\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm"
Jul 30 17:27:07.919098 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919022    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbjjw\" (UniqueName: \"kubernetes.io/projected/8d8ce49e-b875-46ab-8175-7979aa8a1ce5-kube-api-access-zbjjw\") pod \"iptables-alerter-ppvt7\" (UID: \"8d8ce49e-b875-46ab-8175-7979aa8a1ce5\") " pod="openshift-network-operator/iptables-alerter-ppvt7"
Jul 30 17:27:07.919098 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919049    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-log-socket\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:07.919098 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919084    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-host-cni-netd\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:07.919341 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919110    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/642de0d3-59f9-4601-bbc8-94df8e60e3e8-ovnkube-script-lib\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:07.919341 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919127    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 30 17:27:07.919341 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919135    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8d96\" (UniqueName: \"kubernetes.io/projected/642de0d3-59f9-4601-bbc8-94df8e60e3e8-kube-api-access-j8d96\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:07.919341 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919178    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-run-openvswitch\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:07.919341 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919205    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-run-ovn\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:07.919341 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919283    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmw7l\" (UniqueName: \"kubernetes.io/projected/667a3ad7-25b2-430a-9878-94f8e7fa9069-kube-api-access-xmw7l\") pod \"aws-ebs-csi-driver-node-fk7lm\" (UID: \"667a3ad7-25b2-430a-9878-94f8e7fa9069\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm"
Jul 30 17:27:07.919341 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919314    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-host-run-ovn-kubernetes\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:07.919341 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919331    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/667a3ad7-25b2-430a-9878-94f8e7fa9069-registration-dir\") pod \"aws-ebs-csi-driver-node-fk7lm\" (UID: \"667a3ad7-25b2-430a-9878-94f8e7fa9069\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm"
Jul 30 17:27:07.919742 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919352    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d8ce49e-b875-46ab-8175-7979aa8a1ce5-host-slash\") pod \"iptables-alerter-ppvt7\" (UID: \"8d8ce49e-b875-46ab-8175-7979aa8a1ce5\") " pod="openshift-network-operator/iptables-alerter-ppvt7"
Jul 30 17:27:07.919742 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919373    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-host-cni-bin\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:07.919742 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919394    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:07.919742 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919412    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/642de0d3-59f9-4601-bbc8-94df8e60e3e8-ovn-node-metrics-cert\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:07.919742 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919428    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/667a3ad7-25b2-430a-9878-94f8e7fa9069-socket-dir\") pod \"aws-ebs-csi-driver-node-fk7lm\" (UID: \"667a3ad7-25b2-430a-9878-94f8e7fa9069\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm"
Jul 30 17:27:07.919742 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919445    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/667a3ad7-25b2-430a-9878-94f8e7fa9069-etc-selinux\") pod \"aws-ebs-csi-driver-node-fk7lm\" (UID: \"667a3ad7-25b2-430a-9878-94f8e7fa9069\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm"
Jul 30 17:27:07.919742 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919460    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-host-kubelet\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:07.919742 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919476    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-node-log\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:07.919742 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919495    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/667a3ad7-25b2-430a-9878-94f8e7fa9069-device-dir\") pod \"aws-ebs-csi-driver-node-fk7lm\" (UID: \"667a3ad7-25b2-430a-9878-94f8e7fa9069\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm"
Jul 30 17:27:07.919742 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919511    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8d8ce49e-b875-46ab-8175-7979aa8a1ce5-iptables-alerter-script\") pod \"iptables-alerter-ppvt7\" (UID: \"8d8ce49e-b875-46ab-8175-7979aa8a1ce5\") " pod="openshift-network-operator/iptables-alerter-ppvt7"
Jul 30 17:27:07.919742 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919530    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-var-lib-openvswitch\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:07.919742 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919548    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/642de0d3-59f9-4601-bbc8-94df8e60e3e8-ovnkube-config\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:07.919742 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919597    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/642de0d3-59f9-4601-bbc8-94df8e60e3e8-env-overrides\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:07.919742 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919617    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-host-slash\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:07.919742 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919634    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-host-run-netns\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:07.919742 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919671    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-etc-openvswitch\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:07.920523 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.919428    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5tj5c\""
Jul 30 17:27:07.920523 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.920197    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 30 17:27:07.920523 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.920274    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:07.920523 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.920432    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 30 17:27:07.920710 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.920282    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q5r7k\""
Jul 30 17:27:07.921516 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.921493    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 30 17:27:07.921678 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.921654    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-92m2b\""
Jul 30 17:27:07.922031 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.922012    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 30 17:27:07.922196 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.922174    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hkwdn"
Jul 30 17:27:07.922256 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.922221    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 30 17:27:07.922416 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.922399    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 30 17:27:07.922800 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.922781    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nkmvt\""
Jul 30 17:27:07.923062 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.923043    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 30 17:27:07.927414 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.927390    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 30 17:27:07.927524 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.927427    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 30 17:27:07.932510 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.932486    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 30 17:27:07.932719 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.932697    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 30 17:27:07.932843 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.932817    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pnp6q\""
Jul 30 17:27:07.933114 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.933094    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 30 17:27:07.933206 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.933134    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 30 17:27:07.933206 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.933169    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 30 17:27:07.933503 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.933467    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 30 17:27:07.933600 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.933545    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 30 17:27:07.933600 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.933572    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 30 17:27:07.933711 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.933662    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-t7dkp\""
Jul 30 17:27:07.934003 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.933986    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-76sws\""
Jul 30 17:27:07.934103 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.934045    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 30 17:27:07.934181 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.934163    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 30 17:27:07.934322 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.934309    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-x625v\""
Jul 30 17:27:07.945133 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:07.944922    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 17:27:08.007972 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.007947    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 30 17:27:08.009170 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.009146    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 17:27:08.011034 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.011011    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lqlzn"
Jul 30 17:27:08.020324 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020296    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9f46378-294e-4eda-a3c4-08073cac32f2-tmp-dir\") pod \"node-resolver-mxb44\" (UID: \"d9f46378-294e-4eda-a3c4-08073cac32f2\") " pod="openshift-dns/node-resolver-mxb44"
Jul 30 17:27:08.020428 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020340    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c30771bb-82ee-4ecb-a478-3b34bd426bc9-system-cni-dir\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.020428 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020372    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-host-slash\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.020428 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020395    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-host-run-netns\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.020428 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020421    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-host-var-lib-cni-multus\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.020628 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020444    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-etc-systemd\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.020628 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020475    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-sys\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.020628 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020481    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-host-run-netns\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.020628 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020492    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-host-slash\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.020628 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020499    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-var-lib-kubelet\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.020628 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020543    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/667a3ad7-25b2-430a-9878-94f8e7fa9069-sys-fs\") pod \"aws-ebs-csi-driver-node-fk7lm\" (UID: \"667a3ad7-25b2-430a-9878-94f8e7fa9069\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm"
Jul 30 17:27:08.020628 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020578    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnfv6\" (UniqueName: \"kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6\") pod \"network-check-target-vm97k\" (UID: \"98fcc6f5-705d-415c-b08c-87dc618baca2\") " pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:08.020628 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020598    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/667a3ad7-25b2-430a-9878-94f8e7fa9069-sys-fs\") pod \"aws-ebs-csi-driver-node-fk7lm\" (UID: \"667a3ad7-25b2-430a-9878-94f8e7fa9069\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm"
Jul 30 17:27:08.020628 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020612    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-run-systemd\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.020628 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020631    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-system-cni-dir\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.021116 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020653    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-multus-socket-dir-parent\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.021116 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020679    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0cb5d046-0bb1-452a-8562-c46ed626cf89-multus-daemon-config\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.021116 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020708    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-run-systemd\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.021116 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020708    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72g4h\" (UniqueName: \"kubernetes.io/projected/7eb46cea-1c3f-4201-9b43-025d799ca911-kube-api-access-72g4h\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.021116 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020759    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8cdacb83-3cbb-4989-a0d1-9bee40686af3-konnectivity-ca\") pod \"konnectivity-agent-6wttc\" (UID: \"8cdacb83-3cbb-4989-a0d1-9bee40686af3\") " pod="kube-system/konnectivity-agent-6wttc"
Jul 30 17:27:08.021116 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020802    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbjjw\" (UniqueName: \"kubernetes.io/projected/8d8ce49e-b875-46ab-8175-7979aa8a1ce5-kube-api-access-zbjjw\") pod \"iptables-alerter-ppvt7\" (UID: \"8d8ce49e-b875-46ab-8175-7979aa8a1ce5\") " pod="openshift-network-operator/iptables-alerter-ppvt7"
Jul 30 17:27:08.021116 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020916    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-log-socket\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.021116 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020947    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-host-cni-netd\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.021116 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.020974    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8d96\" (UniqueName: \"kubernetes.io/projected/642de0d3-59f9-4601-bbc8-94df8e60e3e8-kube-api-access-j8d96\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.021116 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021006    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0cb5d046-0bb1-452a-8562-c46ed626cf89-cni-binary-copy\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.021116 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021030    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-host-cni-netd\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.021116 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021033    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-log-socket\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.021116 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021032    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-host-run-k8s-cni-cncf-io\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.021116 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021091    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-multus-conf-dir\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.021116 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021117    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8cdacb83-3cbb-4989-a0d1-9bee40686af3-agent-certs\") pod \"konnectivity-agent-6wttc\" (UID: \"8cdacb83-3cbb-4989-a0d1-9bee40686af3\") " pod="kube-system/konnectivity-agent-6wttc"
Jul 30 17:27:08.021773 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021145    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-run-openvswitch\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.021773 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021183    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3251e0d-bc32-44a8-848d-2e64df31b7cc-kubelet-config\") pod \"global-pull-secret-syncer-f4vp7\" (UID: \"a3251e0d-bc32-44a8-848d-2e64df31b7cc\") " pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:08.021773 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021222    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3251e0d-bc32-44a8-848d-2e64df31b7cc-dbus\") pod \"global-pull-secret-syncer-f4vp7\" (UID: \"a3251e0d-bc32-44a8-848d-2e64df31b7cc\") " pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:08.021773 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021245    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-etc-sysconfig\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.021773 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021260    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-run-openvswitch\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.021773 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021284    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c30771bb-82ee-4ecb-a478-3b34bd426bc9-cnibin\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.021773 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021322    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c30771bb-82ee-4ecb-a478-3b34bd426bc9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.021773 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021357    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs\") pod \"network-metrics-daemon-lwrkg\" (UID: \"845cc472-f650-4adc-81b0-94b11e29ecfd\") " pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:08.021773 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021383    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-os-release\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.021773 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021408    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-host-run-multus-certs\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.021773 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021433    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret\") pod \"global-pull-secret-syncer-f4vp7\" (UID: \"a3251e0d-bc32-44a8-848d-2e64df31b7cc\") " pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:08.021773 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021459    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-etc-sysctl-conf\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.021773 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021487    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8267208d-1805-4410-bf0c-595b1f1604f1-host\") pod \"node-ca-hkwdn\" (UID: \"8267208d-1805-4410-bf0c-595b1f1604f1\") " pod="openshift-image-registry/node-ca-hkwdn"
Jul 30 17:27:08.021773 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021515    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c30771bb-82ee-4ecb-a478-3b34bd426bc9-cni-binary-copy\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.021773 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021545    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/667a3ad7-25b2-430a-9878-94f8e7fa9069-registration-dir\") pod \"aws-ebs-csi-driver-node-fk7lm\" (UID: \"667a3ad7-25b2-430a-9878-94f8e7fa9069\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm"
Jul 30 17:27:08.021773 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021569    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d8ce49e-b875-46ab-8175-7979aa8a1ce5-host-slash\") pod \"iptables-alerter-ppvt7\" (UID: \"8d8ce49e-b875-46ab-8175-7979aa8a1ce5\") " pod="openshift-network-operator/iptables-alerter-ppvt7"
Jul 30 17:27:08.022606 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021594    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-host-cni-bin\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.022606 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021617    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-hostroot\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.022606 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021636    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmcz7\" (UniqueName: \"kubernetes.io/projected/c30771bb-82ee-4ecb-a478-3b34bd426bc9-kube-api-access-fmcz7\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.022606 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021653    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/667a3ad7-25b2-430a-9878-94f8e7fa9069-socket-dir\") pod \"aws-ebs-csi-driver-node-fk7lm\" (UID: \"667a3ad7-25b2-430a-9878-94f8e7fa9069\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm"
Jul 30 17:27:08.022606 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021667    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/667a3ad7-25b2-430a-9878-94f8e7fa9069-etc-selinux\") pod \"aws-ebs-csi-driver-node-fk7lm\" (UID: \"667a3ad7-25b2-430a-9878-94f8e7fa9069\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm"
Jul 30 17:27:08.022606 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021689    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm6kp\" (UniqueName: \"kubernetes.io/projected/845cc472-f650-4adc-81b0-94b11e29ecfd-kube-api-access-lm6kp\") pod \"network-metrics-daemon-lwrkg\" (UID: \"845cc472-f650-4adc-81b0-94b11e29ecfd\") " pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:08.022606 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021713    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-host-run-netns\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.022606 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021736    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/667a3ad7-25b2-430a-9878-94f8e7fa9069-device-dir\") pod \"aws-ebs-csi-driver-node-fk7lm\" (UID: \"667a3ad7-25b2-430a-9878-94f8e7fa9069\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm"
Jul 30 17:27:08.022606 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021753    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8d8ce49e-b875-46ab-8175-7979aa8a1ce5-iptables-alerter-script\") pod \"iptables-alerter-ppvt7\" (UID: \"8d8ce49e-b875-46ab-8175-7979aa8a1ce5\") " pod="openshift-network-operator/iptables-alerter-ppvt7"
Jul 30 17:27:08.022606 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021769    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-var-lib-openvswitch\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.022606 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021784    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/642de0d3-59f9-4601-bbc8-94df8e60e3e8-env-overrides\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.022606 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021798    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8267208d-1805-4410-bf0c-595b1f1604f1-serviceca\") pod \"node-ca-hkwdn\" (UID: \"8267208d-1805-4410-bf0c-595b1f1604f1\") " pod="openshift-image-registry/node-ca-hkwdn"
Jul 30 17:27:08.022606 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021817    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-etc-openvswitch\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.022606 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021857    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-cnibin\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.022606 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021911    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-etc-kubernetes\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.022606 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.021977    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/667a3ad7-25b2-430a-9878-94f8e7fa9069-device-dir\") pod \"aws-ebs-csi-driver-node-fk7lm\" (UID: \"667a3ad7-25b2-430a-9878-94f8e7fa9069\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm"
Jul 30 17:27:08.022606 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.022028    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/667a3ad7-25b2-430a-9878-94f8e7fa9069-etc-selinux\") pod \"aws-ebs-csi-driver-node-fk7lm\" (UID: \"667a3ad7-25b2-430a-9878-94f8e7fa9069\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm"
Jul 30 17:27:08.023497 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.022048    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-etc-openvswitch\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.023497 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.022093    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-var-lib-openvswitch\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.023497 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.022135    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/667a3ad7-25b2-430a-9878-94f8e7fa9069-socket-dir\") pod \"aws-ebs-csi-driver-node-fk7lm\" (UID: \"667a3ad7-25b2-430a-9878-94f8e7fa9069\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm"
Jul 30 17:27:08.023497 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.022626    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8d8ce49e-b875-46ab-8175-7979aa8a1ce5-iptables-alerter-script\") pod \"iptables-alerter-ppvt7\" (UID: \"8d8ce49e-b875-46ab-8175-7979aa8a1ce5\") " pod="openshift-network-operator/iptables-alerter-ppvt7"
Jul 30 17:27:08.023497 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.022674    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/642de0d3-59f9-4601-bbc8-94df8e60e3e8-env-overrides\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.023497 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.022733    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/667a3ad7-25b2-430a-9878-94f8e7fa9069-registration-dir\") pod \"aws-ebs-csi-driver-node-fk7lm\" (UID: \"667a3ad7-25b2-430a-9878-94f8e7fa9069\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm"
Jul 30 17:27:08.023497 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.022739    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-systemd-units\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.023497 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.022770    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-etc-modprobe-d\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.023497 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.022774    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-host-cni-bin\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.023497 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.022797    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-lib-modules\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.023497 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.022812    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d8ce49e-b875-46ab-8175-7979aa8a1ce5-host-slash\") pod \"iptables-alerter-ppvt7\" (UID: \"8d8ce49e-b875-46ab-8175-7979aa8a1ce5\") " pod="openshift-network-operator/iptables-alerter-ppvt7"
Jul 30 17:27:08.023497 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.022826    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/667a3ad7-25b2-430a-9878-94f8e7fa9069-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fk7lm\" (UID: \"667a3ad7-25b2-430a-9878-94f8e7fa9069\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm"
Jul 30 17:27:08.023497 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.022868    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-systemd-units\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.023497 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.022877    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/642de0d3-59f9-4601-bbc8-94df8e60e3e8-ovnkube-script-lib\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.023497 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.022950    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-etc-kubernetes\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.023497 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.022956    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/667a3ad7-25b2-430a-9878-94f8e7fa9069-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fk7lm\" (UID: \"667a3ad7-25b2-430a-9878-94f8e7fa9069\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm"
Jul 30 17:27:08.023497 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023006    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hkqz\" (UniqueName: \"kubernetes.io/projected/8267208d-1805-4410-bf0c-595b1f1604f1-kube-api-access-9hkqz\") pod \"node-ca-hkwdn\" (UID: \"8267208d-1805-4410-bf0c-595b1f1604f1\") " pod="openshift-image-registry/node-ca-hkwdn"
Jul 30 17:27:08.024398 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023044    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-run-ovn\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.024398 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023084    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-multus-cni-dir\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.024398 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023111    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnmdw\" (UniqueName: \"kubernetes.io/projected/d9f46378-294e-4eda-a3c4-08073cac32f2-kube-api-access-pnmdw\") pod \"node-resolver-mxb44\" (UID: \"d9f46378-294e-4eda-a3c4-08073cac32f2\") " pod="openshift-dns/node-resolver-mxb44"
Jul 30 17:27:08.024398 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023113    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lqlzn"
Jul 30 17:27:08.024398 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023151    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-run-ovn\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.024398 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023183    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c30771bb-82ee-4ecb-a478-3b34bd426bc9-os-release\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.024398 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023221    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c30771bb-82ee-4ecb-a478-3b34bd426bc9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.024398 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023253    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmw7l\" (UniqueName: \"kubernetes.io/projected/667a3ad7-25b2-430a-9878-94f8e7fa9069-kube-api-access-xmw7l\") pod \"aws-ebs-csi-driver-node-fk7lm\" (UID: \"667a3ad7-25b2-430a-9878-94f8e7fa9069\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm"
Jul 30 17:27:08.024398 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023281    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-host-run-ovn-kubernetes\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.024398 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023299    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-host-var-lib-cni-bin\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.024398 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023331    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d9f46378-294e-4eda-a3c4-08073cac32f2-hosts-file\") pod \"node-resolver-mxb44\" (UID: \"d9f46378-294e-4eda-a3c4-08073cac32f2\") " pod="openshift-dns/node-resolver-mxb44"
Jul 30 17:27:08.024398 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023368    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-host-run-ovn-kubernetes\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.024398 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023403    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-etc-sysctl-d\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.024398 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023425    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-run\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.024398 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023450    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c30771bb-82ee-4ecb-a478-3b34bd426bc9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.024398 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023470    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.024398 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023495    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.025277 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023509    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/642de0d3-59f9-4601-bbc8-94df8e60e3e8-ovn-node-metrics-cert\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.025277 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023522    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/642de0d3-59f9-4601-bbc8-94df8e60e3e8-ovnkube-script-lib\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.025277 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023569    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-host\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.025277 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023594    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7eb46cea-1c3f-4201-9b43-025d799ca911-etc-tuned\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.025277 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023683    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7eb46cea-1c3f-4201-9b43-025d799ca911-tmp\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.025277 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023719    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-host-kubelet\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.025277 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023745    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-node-log\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.025277 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023770    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-host-var-lib-kubelet\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.025277 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023778    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-host-kubelet\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.025277 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023797    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/642de0d3-59f9-4601-bbc8-94df8e60e3e8-ovnkube-config\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.025277 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023804    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/642de0d3-59f9-4601-bbc8-94df8e60e3e8-node-log\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.025277 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023878    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghdn5\" (UniqueName: \"kubernetes.io/projected/0cb5d046-0bb1-452a-8562-c46ed626cf89-kube-api-access-ghdn5\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.025277 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.023909    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 30 17:27:08.025277 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.024336    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/642de0d3-59f9-4601-bbc8-94df8e60e3e8-ovnkube-config\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.028255 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.028231    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/642de0d3-59f9-4601-bbc8-94df8e60e3e8-ovn-node-metrics-cert\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.029856 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:08.029834    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 17:27:08.029980 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:08.029880    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 17:27:08.029980 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:08.029919    2577 projected.go:196] Error preparing data for projected volume kube-api-access-fnfv6 for pod openshift-network-diagnostics/network-check-target-vm97k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 17:27:08.030100 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:08.030009    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6 podName:98fcc6f5-705d-415c-b08c-87dc618baca2 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:08.529973111 +0000 UTC m=+2.983537099 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fnfv6" (UniqueName: "kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6") pod "network-check-target-vm97k" (UID: "98fcc6f5-705d-415c-b08c-87dc618baca2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 17:27:08.033276 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.033252    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmw7l\" (UniqueName: \"kubernetes.io/projected/667a3ad7-25b2-430a-9878-94f8e7fa9069-kube-api-access-xmw7l\") pod \"aws-ebs-csi-driver-node-fk7lm\" (UID: \"667a3ad7-25b2-430a-9878-94f8e7fa9069\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm"
Jul 30 17:27:08.035295 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.035269    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8d96\" (UniqueName: \"kubernetes.io/projected/642de0d3-59f9-4601-bbc8-94df8e60e3e8-kube-api-access-j8d96\") pod \"ovnkube-node-b6r8l\" (UID: \"642de0d3-59f9-4601-bbc8-94df8e60e3e8\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.042714 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.042694    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbjjw\" (UniqueName: \"kubernetes.io/projected/8d8ce49e-b875-46ab-8175-7979aa8a1ce5-kube-api-access-zbjjw\") pod \"iptables-alerter-ppvt7\" (UID: \"8d8ce49e-b875-46ab-8175-7979aa8a1ce5\") " pod="openshift-network-operator/iptables-alerter-ppvt7"
Jul 30 17:27:08.109443 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.109382    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-203.ec2.internal" event={"ID":"ebe8f1131fdea96935597faec1e85919","Type":"ContainerStarted","Data":"9cd304e0e1c307810be38e774ed1a064bc4ff5c09469ef37a91d15b15faa191a"}
Jul 30 17:27:08.111112 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.111083    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-203.ec2.internal" event={"ID":"64aa76e700834f6545b825fca32978ba","Type":"ContainerStarted","Data":"eaedc5ae09cd858eda1b7c64f02cfdb158095207e9bdbac4d055d8f0f49a957c"}
Jul 30 17:27:08.124768 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.124740    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-cnibin\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.124919 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.124778    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-etc-kubernetes\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.124919 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.124798    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-etc-modprobe-d\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.124919 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.124822    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-lib-modules\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.124919 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.124849    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-etc-kubernetes\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.124919 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.124882    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hkqz\" (UniqueName: \"kubernetes.io/projected/8267208d-1805-4410-bf0c-595b1f1604f1-kube-api-access-9hkqz\") pod \"node-ca-hkwdn\" (UID: \"8267208d-1805-4410-bf0c-595b1f1604f1\") " pod="openshift-image-registry/node-ca-hkwdn"
Jul 30 17:27:08.125203 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.124921    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-multus-cni-dir\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.125203 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.124969    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnmdw\" (UniqueName: \"kubernetes.io/projected/d9f46378-294e-4eda-a3c4-08073cac32f2-kube-api-access-pnmdw\") pod \"node-resolver-mxb44\" (UID: \"d9f46378-294e-4eda-a3c4-08073cac32f2\") " pod="openshift-dns/node-resolver-mxb44"
Jul 30 17:27:08.125203 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125016    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c30771bb-82ee-4ecb-a478-3b34bd426bc9-os-release\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.125203 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125039    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c30771bb-82ee-4ecb-a478-3b34bd426bc9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.125203 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125072    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-etc-modprobe-d\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.125203 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125072    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-cnibin\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.125203 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125147    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-host-var-lib-cni-bin\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.125203 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125171    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c30771bb-82ee-4ecb-a478-3b34bd426bc9-os-release\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.125610 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125217    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-multus-cni-dir\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.125610 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125328    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-lib-modules\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.125610 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125387    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-etc-kubernetes\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.125610 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125427    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-etc-kubernetes\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.125610 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125090    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-host-var-lib-cni-bin\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.125610 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125500    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d9f46378-294e-4eda-a3c4-08073cac32f2-hosts-file\") pod \"node-resolver-mxb44\" (UID: \"d9f46378-294e-4eda-a3c4-08073cac32f2\") " pod="openshift-dns/node-resolver-mxb44"
Jul 30 17:27:08.125610 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125535    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-etc-sysctl-d\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.125610 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125559    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-run\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.125610 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125583    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c30771bb-82ee-4ecb-a478-3b34bd426bc9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.125610 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125610    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-host\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.126141 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125647    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7eb46cea-1c3f-4201-9b43-025d799ca911-etc-tuned\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.126141 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125679    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7eb46cea-1c3f-4201-9b43-025d799ca911-tmp\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.126141 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125705    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-host-var-lib-kubelet\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.126141 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125734    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-etc-sysctl-d\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.126141 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125734    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghdn5\" (UniqueName: \"kubernetes.io/projected/0cb5d046-0bb1-452a-8562-c46ed626cf89-kube-api-access-ghdn5\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.126141 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125781    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9f46378-294e-4eda-a3c4-08073cac32f2-tmp-dir\") pod \"node-resolver-mxb44\" (UID: \"d9f46378-294e-4eda-a3c4-08073cac32f2\") " pod="openshift-dns/node-resolver-mxb44"
Jul 30 17:27:08.126141 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125791    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d9f46378-294e-4eda-a3c4-08073cac32f2-hosts-file\") pod \"node-resolver-mxb44\" (UID: \"d9f46378-294e-4eda-a3c4-08073cac32f2\") " pod="openshift-dns/node-resolver-mxb44"
Jul 30 17:27:08.126141 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125794    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c30771bb-82ee-4ecb-a478-3b34bd426bc9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.126141 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125805    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c30771bb-82ee-4ecb-a478-3b34bd426bc9-system-cni-dir\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.126141 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125859    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-host-var-lib-cni-multus\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.126141 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125914    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-etc-systemd\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.126141 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125944    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-sys\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.126141 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.125969    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-var-lib-kubelet\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.126141 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126013    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-system-cni-dir\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.126141 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126042    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-multus-socket-dir-parent\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.126141 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126071    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0cb5d046-0bb1-452a-8562-c46ed626cf89-multus-daemon-config\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.126141 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126101    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72g4h\" (UniqueName: \"kubernetes.io/projected/7eb46cea-1c3f-4201-9b43-025d799ca911-kube-api-access-72g4h\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.126141 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126127    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8cdacb83-3cbb-4989-a0d1-9bee40686af3-konnectivity-ca\") pod \"konnectivity-agent-6wttc\" (UID: \"8cdacb83-3cbb-4989-a0d1-9bee40686af3\") " pod="kube-system/konnectivity-agent-6wttc"
Jul 30 17:27:08.127081 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126157    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0cb5d046-0bb1-452a-8562-c46ed626cf89-cni-binary-copy\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.127081 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126181    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-host-run-k8s-cni-cncf-io\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.127081 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126208    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-multus-conf-dir\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.127081 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126232    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8cdacb83-3cbb-4989-a0d1-9bee40686af3-agent-certs\") pod \"konnectivity-agent-6wttc\" (UID: \"8cdacb83-3cbb-4989-a0d1-9bee40686af3\") " pod="kube-system/konnectivity-agent-6wttc"
Jul 30 17:27:08.127081 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126259    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3251e0d-bc32-44a8-848d-2e64df31b7cc-kubelet-config\") pod \"global-pull-secret-syncer-f4vp7\" (UID: \"a3251e0d-bc32-44a8-848d-2e64df31b7cc\") " pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:08.127081 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126285    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3251e0d-bc32-44a8-848d-2e64df31b7cc-dbus\") pod \"global-pull-secret-syncer-f4vp7\" (UID: \"a3251e0d-bc32-44a8-848d-2e64df31b7cc\") " pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:08.127081 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126325    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-etc-sysconfig\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.127081 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126407    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c30771bb-82ee-4ecb-a478-3b34bd426bc9-cnibin\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.127081 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126437    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c30771bb-82ee-4ecb-a478-3b34bd426bc9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.127081 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126465    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs\") pod \"network-metrics-daemon-lwrkg\" (UID: \"845cc472-f650-4adc-81b0-94b11e29ecfd\") " pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:08.127081 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126513    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-os-release\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.127081 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126542    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-host-run-multus-certs\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.127081 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126569    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret\") pod \"global-pull-secret-syncer-f4vp7\" (UID: \"a3251e0d-bc32-44a8-848d-2e64df31b7cc\") " pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:08.127081 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-etc-sysctl-conf\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.127081 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126625    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8267208d-1805-4410-bf0c-595b1f1604f1-host\") pod \"node-ca-hkwdn\" (UID: \"8267208d-1805-4410-bf0c-595b1f1604f1\") " pod="openshift-image-registry/node-ca-hkwdn"
Jul 30 17:27:08.127081 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126650    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c30771bb-82ee-4ecb-a478-3b34bd426bc9-cni-binary-copy\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.127081 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126697    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-hostroot\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.127951 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126727    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmcz7\" (UniqueName: \"kubernetes.io/projected/c30771bb-82ee-4ecb-a478-3b34bd426bc9-kube-api-access-fmcz7\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.127951 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126760    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm6kp\" (UniqueName: \"kubernetes.io/projected/845cc472-f650-4adc-81b0-94b11e29ecfd-kube-api-access-lm6kp\") pod \"network-metrics-daemon-lwrkg\" (UID: \"845cc472-f650-4adc-81b0-94b11e29ecfd\") " pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:08.127951 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126785    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-host-run-netns\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.127951 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.126821    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8267208d-1805-4410-bf0c-595b1f1604f1-serviceca\") pod \"node-ca-hkwdn\" (UID: \"8267208d-1805-4410-bf0c-595b1f1604f1\") " pod="openshift-image-registry/node-ca-hkwdn"
Jul 30 17:27:08.127951 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.127255    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c30771bb-82ee-4ecb-a478-3b34bd426bc9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.127951 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.127339    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-host\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.127951 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.127374    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c30771bb-82ee-4ecb-a478-3b34bd426bc9-system-cni-dir\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.127951 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.127387    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8267208d-1805-4410-bf0c-595b1f1604f1-serviceca\") pod \"node-ca-hkwdn\" (UID: \"8267208d-1805-4410-bf0c-595b1f1604f1\") " pod="openshift-image-registry/node-ca-hkwdn"
Jul 30 17:27:08.127951 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.127448    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-host-var-lib-cni-multus\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.127951 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.127492    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-etc-systemd\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.127951 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.127549    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-sys\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.127951 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.127597    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-var-lib-kubelet\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.127951 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.127640    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-system-cni-dir\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.127951 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.127672    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-multus-socket-dir-parent\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.127951 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:08.127955    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 17:27:08.130121 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:08.128005    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs podName:845cc472-f650-4adc-81b0-94b11e29ecfd nodeName:}" failed. No retries permitted until 2026-07-30 17:27:08.627990417 +0000 UTC m=+3.081554388 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs") pod "network-metrics-daemon-lwrkg" (UID: "845cc472-f650-4adc-81b0-94b11e29ecfd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 17:27:08.130121 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.128109    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9f46378-294e-4eda-a3c4-08073cac32f2-tmp-dir\") pod \"node-resolver-mxb44\" (UID: \"d9f46378-294e-4eda-a3c4-08073cac32f2\") " pod="openshift-dns/node-resolver-mxb44"
Jul 30 17:27:08.130121 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.128160    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0cb5d046-0bb1-452a-8562-c46ed626cf89-multus-daemon-config\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.130121 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.128198    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8267208d-1805-4410-bf0c-595b1f1604f1-host\") pod \"node-ca-hkwdn\" (UID: \"8267208d-1805-4410-bf0c-595b1f1604f1\") " pod="openshift-image-registry/node-ca-hkwdn"
Jul 30 17:27:08.130121 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.128240    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3251e0d-bc32-44a8-848d-2e64df31b7cc-kubelet-config\") pod \"global-pull-secret-syncer-f4vp7\" (UID: \"a3251e0d-bc32-44a8-848d-2e64df31b7cc\") " pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:08.130121 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.128279    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-os-release\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.130121 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.128295    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3251e0d-bc32-44a8-848d-2e64df31b7cc-dbus\") pod \"global-pull-secret-syncer-f4vp7\" (UID: \"a3251e0d-bc32-44a8-848d-2e64df31b7cc\") " pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:08.130121 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.128327    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-host-run-multus-certs\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.130121 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.128348    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-etc-sysconfig\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.130121 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.128386    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c30771bb-82ee-4ecb-a478-3b34bd426bc9-cnibin\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.130121 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:08.128518    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 17:27:08.130121 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.128540    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-etc-sysctl-conf\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.130121 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.128556    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c30771bb-82ee-4ecb-a478-3b34bd426bc9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.130121 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:08.128564    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret podName:a3251e0d-bc32-44a8-848d-2e64df31b7cc nodeName:}" failed. No retries permitted until 2026-07-30 17:27:08.628548052 +0000 UTC m=+3.082112024 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret") pod "global-pull-secret-syncer-f4vp7" (UID: "a3251e0d-bc32-44a8-848d-2e64df31b7cc") : object "kube-system"/"original-pull-secret" not registered
Jul 30 17:27:08.130121 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.128653    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7eb46cea-1c3f-4201-9b43-025d799ca911-run\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.130121 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.128702    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-hostroot\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.130121 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.128673    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c30771bb-82ee-4ecb-a478-3b34bd426bc9-cni-binary-copy\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.130880 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.128759    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-host-var-lib-kubelet\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.130880 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.128854    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-host-run-netns\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.130880 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.128942    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-host-run-k8s-cni-cncf-io\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.130880 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.129180    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7eb46cea-1c3f-4201-9b43-025d799ca911-tmp\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.130880 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.129201    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7eb46cea-1c3f-4201-9b43-025d799ca911-etc-tuned\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.130880 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.129432    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0cb5d046-0bb1-452a-8562-c46ed626cf89-multus-conf-dir\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.130880 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.129605    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8cdacb83-3cbb-4989-a0d1-9bee40686af3-konnectivity-ca\") pod \"konnectivity-agent-6wttc\" (UID: \"8cdacb83-3cbb-4989-a0d1-9bee40686af3\") " pod="kube-system/konnectivity-agent-6wttc"
Jul 30 17:27:08.130880 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.129605    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0cb5d046-0bb1-452a-8562-c46ed626cf89-cni-binary-copy\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.132489 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.132164    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8cdacb83-3cbb-4989-a0d1-9bee40686af3-agent-certs\") pod \"konnectivity-agent-6wttc\" (UID: \"8cdacb83-3cbb-4989-a0d1-9bee40686af3\") " pod="kube-system/konnectivity-agent-6wttc"
Jul 30 17:27:08.147938 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.147901    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm6kp\" (UniqueName: \"kubernetes.io/projected/845cc472-f650-4adc-81b0-94b11e29ecfd-kube-api-access-lm6kp\") pod \"network-metrics-daemon-lwrkg\" (UID: \"845cc472-f650-4adc-81b0-94b11e29ecfd\") " pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:08.156616 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.156585    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72g4h\" (UniqueName: \"kubernetes.io/projected/7eb46cea-1c3f-4201-9b43-025d799ca911-kube-api-access-72g4h\") pod \"tuned-sfrfh\" (UID: \"7eb46cea-1c3f-4201-9b43-025d799ca911\") " pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.157117 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.157091    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hkqz\" (UniqueName: \"kubernetes.io/projected/8267208d-1805-4410-bf0c-595b1f1604f1-kube-api-access-9hkqz\") pod \"node-ca-hkwdn\" (UID: \"8267208d-1805-4410-bf0c-595b1f1604f1\") " pod="openshift-image-registry/node-ca-hkwdn"
Jul 30 17:27:08.157219 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.157139    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmcz7\" (UniqueName: \"kubernetes.io/projected/c30771bb-82ee-4ecb-a478-3b34bd426bc9-kube-api-access-fmcz7\") pod \"multus-additional-cni-plugins-ctb9m\" (UID: \"c30771bb-82ee-4ecb-a478-3b34bd426bc9\") " pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.157956 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.157931    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghdn5\" (UniqueName: \"kubernetes.io/projected/0cb5d046-0bb1-452a-8562-c46ed626cf89-kube-api-access-ghdn5\") pod \"multus-9nsjh\" (UID: \"0cb5d046-0bb1-452a-8562-c46ed626cf89\") " pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.158262 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.158238    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnmdw\" (UniqueName: \"kubernetes.io/projected/d9f46378-294e-4eda-a3c4-08073cac32f2-kube-api-access-pnmdw\") pod \"node-resolver-mxb44\" (UID: \"d9f46378-294e-4eda-a3c4-08073cac32f2\") " pod="openshift-dns/node-resolver-mxb44"
Jul 30 17:27:08.224270 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.224238    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm"
Jul 30 17:27:08.238495 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.238467    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ppvt7"
Jul 30 17:27:08.249814 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.249792    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:08.256642 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.256624    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6wttc"
Jul 30 17:27:08.265935 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.265768    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mxb44"
Jul 30 17:27:08.274634 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.274607    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9nsjh"
Jul 30 17:27:08.281943 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.281926    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ctb9m"
Jul 30 17:27:08.290709 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.290682    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sfrfh"
Jul 30 17:27:08.298521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.298494    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hkwdn"
Jul 30 17:27:08.630345 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.630244    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnfv6\" (UniqueName: \"kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6\") pod \"network-check-target-vm97k\" (UID: \"98fcc6f5-705d-415c-b08c-87dc618baca2\") " pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:08.630345 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.630309    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs\") pod \"network-metrics-daemon-lwrkg\" (UID: \"845cc472-f650-4adc-81b0-94b11e29ecfd\") " pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:08.630575 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:08.630381    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret\") pod \"global-pull-secret-syncer-f4vp7\" (UID: \"a3251e0d-bc32-44a8-848d-2e64df31b7cc\") " pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:08.630575 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:08.630428    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 17:27:08.630575 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:08.630496    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs podName:845cc472-f650-4adc-81b0-94b11e29ecfd nodeName:}" failed. No retries permitted until 2026-07-30 17:27:09.630476963 +0000 UTC m=+4.084040933 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs") pod "network-metrics-daemon-lwrkg" (UID: "845cc472-f650-4adc-81b0-94b11e29ecfd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 17:27:08.630575 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:08.630545    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 17:27:08.630575 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:08.630573    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 17:27:08.630822 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:08.630592    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 17:27:08.630822 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:08.630600    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret podName:a3251e0d-bc32-44a8-848d-2e64df31b7cc nodeName:}" failed. No retries permitted until 2026-07-30 17:27:09.63058366 +0000 UTC m=+4.084147634 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret") pod "global-pull-secret-syncer-f4vp7" (UID: "a3251e0d-bc32-44a8-848d-2e64df31b7cc") : object "kube-system"/"original-pull-secret" not registered
Jul 30 17:27:08.630822 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:08.630602    2577 projected.go:196] Error preparing data for projected volume kube-api-access-fnfv6 for pod openshift-network-diagnostics/network-check-target-vm97k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 17:27:08.630822 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:08.630666    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6 podName:98fcc6f5-705d-415c-b08c-87dc618baca2 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:09.630652644 +0000 UTC m=+4.084216630 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fnfv6" (UniqueName: "kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6") pod "network-check-target-vm97k" (UID: "98fcc6f5-705d-415c-b08c-87dc618baca2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 17:27:09.027022 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:09.026922    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 17:22:08 +0000 UTC" deadline="2028-04-07 20:26:01.902443382 +0000 UTC"
Jul 30 17:27:09.027022 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:09.026973    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14810h58m52.875473758s"
Jul 30 17:27:09.104719 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:09.104682    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:09.104906 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:09.104817    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4vp7" podUID="a3251e0d-bc32-44a8-848d-2e64df31b7cc"
Jul 30 17:27:09.104906 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:09.104876    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:09.105062 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:09.105039    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwrkg" podUID="845cc472-f650-4adc-81b0-94b11e29ecfd"
Jul 30 17:27:09.239127 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:09.239095    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc30771bb_82ee_4ecb_a478_3b34bd426bc9.slice/crio-e879da76d341f5e0c2e3f4eb976289c516c731006d467add7fb240fb65c76e7a WatchSource:0}: Error finding container e879da76d341f5e0c2e3f4eb976289c516c731006d467add7fb240fb65c76e7a: Status 404 returned error can't find the container with id e879da76d341f5e0c2e3f4eb976289c516c731006d467add7fb240fb65c76e7a
Jul 30 17:27:09.262540 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:09.259345    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9f46378_294e_4eda_a3c4_08073cac32f2.slice/crio-80c38401ff580b87e8563f0734587b2200400c7aea04a390b6124fed75597562 WatchSource:0}: Error finding container 80c38401ff580b87e8563f0734587b2200400c7aea04a390b6124fed75597562: Status 404 returned error can't find the container with id 80c38401ff580b87e8563f0734587b2200400c7aea04a390b6124fed75597562
Jul 30 17:27:09.271457 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:09.271427    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cdacb83_3cbb_4989_a0d1_9bee40686af3.slice/crio-42cc3da0bb49576bd907219125d3b367b7f3d58515f334c170e80cd3d4b1bdd2 WatchSource:0}: Error finding container 42cc3da0bb49576bd907219125d3b367b7f3d58515f334c170e80cd3d4b1bdd2: Status 404 returned error can't find the container with id 42cc3da0bb49576bd907219125d3b367b7f3d58515f334c170e80cd3d4b1bdd2
Jul 30 17:27:09.272344 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:09.271869    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8267208d_1805_4410_bf0c_595b1f1604f1.slice/crio-ca9495d3d77fff51dd642bec40b2c648b5256c9980d6839875df759c5cd1f3fc WatchSource:0}: Error finding container ca9495d3d77fff51dd642bec40b2c648b5256c9980d6839875df759c5cd1f3fc: Status 404 returned error can't find the container with id ca9495d3d77fff51dd642bec40b2c648b5256c9980d6839875df759c5cd1f3fc
Jul 30 17:27:09.638843 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:09.638545    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnfv6\" (UniqueName: \"kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6\") pod \"network-check-target-vm97k\" (UID: \"98fcc6f5-705d-415c-b08c-87dc618baca2\") " pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:09.639117 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:09.638901    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs\") pod \"network-metrics-daemon-lwrkg\" (UID: \"845cc472-f650-4adc-81b0-94b11e29ecfd\") " pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:09.639117 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:09.638774    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 17:27:09.639117 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:09.638941    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 17:27:09.639117 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:09.638957    2577 projected.go:196] Error preparing data for projected volume kube-api-access-fnfv6 for pod openshift-network-diagnostics/network-check-target-vm97k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 17:27:09.639117 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:09.639032    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6 podName:98fcc6f5-705d-415c-b08c-87dc618baca2 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:11.639010914 +0000 UTC m=+6.092574906 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fnfv6" (UniqueName: "kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6") pod "network-check-target-vm97k" (UID: "98fcc6f5-705d-415c-b08c-87dc618baca2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 17:27:09.639117 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:09.639039    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 17:27:09.639117 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:09.639078    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret podName:a3251e0d-bc32-44a8-848d-2e64df31b7cc nodeName:}" failed. No retries permitted until 2026-07-30 17:27:11.639066732 +0000 UTC m=+6.092630716 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret") pod "global-pull-secret-syncer-f4vp7" (UID: "a3251e0d-bc32-44a8-848d-2e64df31b7cc") : object "kube-system"/"original-pull-secret" not registered
Jul 30 17:27:09.639422 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:09.639135    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 17:27:09.639422 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:09.639167    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs podName:845cc472-f650-4adc-81b0-94b11e29ecfd nodeName:}" failed. No retries permitted until 2026-07-30 17:27:11.639155937 +0000 UTC m=+6.092719924 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs") pod "network-metrics-daemon-lwrkg" (UID: "845cc472-f650-4adc-81b0-94b11e29ecfd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 17:27:09.639422 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:09.638938    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret\") pod \"global-pull-secret-syncer-f4vp7\" (UID: \"a3251e0d-bc32-44a8-848d-2e64df31b7cc\") " pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:10.027484 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:10.027364    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 17:22:08 +0000 UTC" deadline="2028-04-17 20:56:44.326379564 +0000 UTC"
Jul 30 17:27:10.027484 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:10.027431    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15051h29m34.298952395s"
Jul 30 17:27:10.106142 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:10.106112    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:10.106329 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:10.106234    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vm97k" podUID="98fcc6f5-705d-415c-b08c-87dc618baca2"
Jul 30 17:27:10.124582 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:10.124541    2577 generic.go:356] "Generic (PLEG): container finished" podID="64aa76e700834f6545b825fca32978ba" containerID="9aa85475975b4bf2b88fa57bc218464d4943768c450a5d65bffd78c426db2c29" exitCode=0
Jul 30 17:27:10.124780 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:10.124643    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-203.ec2.internal" event={"ID":"64aa76e700834f6545b825fca32978ba","Type":"ContainerDied","Data":"9aa85475975b4bf2b88fa57bc218464d4943768c450a5d65bffd78c426db2c29"}
Jul 30 17:27:10.143471 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:10.143439    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6wttc" event={"ID":"8cdacb83-3cbb-4989-a0d1-9bee40686af3","Type":"ContainerStarted","Data":"42cc3da0bb49576bd907219125d3b367b7f3d58515f334c170e80cd3d4b1bdd2"}
Jul 30 17:27:10.149117 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:10.149088    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mxb44" event={"ID":"d9f46378-294e-4eda-a3c4-08073cac32f2","Type":"ContainerStarted","Data":"80c38401ff580b87e8563f0734587b2200400c7aea04a390b6124fed75597562"}
Jul 30 17:27:10.186448 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:10.186402    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ctb9m" event={"ID":"c30771bb-82ee-4ecb-a478-3b34bd426bc9","Type":"ContainerStarted","Data":"e879da76d341f5e0c2e3f4eb976289c516c731006d467add7fb240fb65c76e7a"}
Jul 30 17:27:10.214931 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:10.214805    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hkwdn" event={"ID":"8267208d-1805-4410-bf0c-595b1f1604f1","Type":"ContainerStarted","Data":"ca9495d3d77fff51dd642bec40b2c648b5256c9980d6839875df759c5cd1f3fc"}
Jul 30 17:27:10.239336 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:10.239288    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ppvt7" event={"ID":"8d8ce49e-b875-46ab-8175-7979aa8a1ce5","Type":"ContainerStarted","Data":"6fd2aae9b2bc99729f6675f3ecf7f217b71b973729479d1dd4134082f970a493"}
Jul 30 17:27:10.244060 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:10.244027    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sfrfh" event={"ID":"7eb46cea-1c3f-4201-9b43-025d799ca911","Type":"ContainerStarted","Data":"a95f8d93ef90a49df585c830f66b904270452312549a5d275605f28dfda58f9e"}
Jul 30 17:27:10.257659 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:10.257623    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm" event={"ID":"667a3ad7-25b2-430a-9878-94f8e7fa9069","Type":"ContainerStarted","Data":"10f8a804b6639f6f4b2d8f573c04f39cb1f0152eb90b0a2007f6dae6dd488ee1"}
Jul 30 17:27:10.278123 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:10.277958    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l" event={"ID":"642de0d3-59f9-4601-bbc8-94df8e60e3e8","Type":"ContainerStarted","Data":"ad86a33c1cf6e502f99d1044829058b83af4b330515e8601f225c8634d6f3561"}
Jul 30 17:27:10.279849 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:10.279808    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9nsjh" event={"ID":"0cb5d046-0bb1-452a-8562-c46ed626cf89","Type":"ContainerStarted","Data":"5c2751f876300663d96858af5d517c0f0976e43ad97bf44594aafa7ff3caf7f2"}
Jul 30 17:27:10.293285 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:10.293164    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-203.ec2.internal" event={"ID":"ebe8f1131fdea96935597faec1e85919","Type":"ContainerStarted","Data":"0907face7b7b70b8384e4dff6d81982f0d11d53fd1fcfee0ea706cdf63151125"}
Jul 30 17:27:11.104661 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:11.104624    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:11.105151 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:11.104667    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:11.105151 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:11.104770    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4vp7" podUID="a3251e0d-bc32-44a8-848d-2e64df31b7cc"
Jul 30 17:27:11.105151 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:11.104946    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwrkg" podUID="845cc472-f650-4adc-81b0-94b11e29ecfd"
Jul 30 17:27:11.322517 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:11.322465    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-203.ec2.internal" event={"ID":"64aa76e700834f6545b825fca32978ba","Type":"ContainerStarted","Data":"6f327c90bc4ca113776a4668bc268b913a7a5ba3e2eb362e47a315e12a92bb97"}
Jul 30 17:27:11.342422 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:11.342357    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-203.ec2.internal" podStartSLOduration=3.342333046 podStartE2EDuration="3.342333046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:27:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:10.315210386 +0000 UTC m=+4.768774382" watchObservedRunningTime="2026-07-30 17:27:11.342333046 +0000 UTC m=+5.795897042"
Jul 30 17:27:11.343046 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:11.342992    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-203.ec2.internal" podStartSLOduration=4.342979211 podStartE2EDuration="4.342979211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:27:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:11.342240212 +0000 UTC m=+5.795804210" watchObservedRunningTime="2026-07-30 17:27:11.342979211 +0000 UTC m=+5.796543205"
Jul 30 17:27:11.657434 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:11.657389    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnfv6\" (UniqueName: \"kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6\") pod \"network-check-target-vm97k\" (UID: \"98fcc6f5-705d-415c-b08c-87dc618baca2\") " pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:11.657724 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:11.657461    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs\") pod \"network-metrics-daemon-lwrkg\" (UID: \"845cc472-f650-4adc-81b0-94b11e29ecfd\") " pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:11.657724 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:11.657493    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret\") pod \"global-pull-secret-syncer-f4vp7\" (UID: \"a3251e0d-bc32-44a8-848d-2e64df31b7cc\") " pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:11.657724 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:11.657640    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 17:27:11.657724 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:11.657704    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret podName:a3251e0d-bc32-44a8-848d-2e64df31b7cc nodeName:}" failed. No retries permitted until 2026-07-30 17:27:15.657685358 +0000 UTC m=+10.111249332 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret") pod "global-pull-secret-syncer-f4vp7" (UID: "a3251e0d-bc32-44a8-848d-2e64df31b7cc") : object "kube-system"/"original-pull-secret" not registered
Jul 30 17:27:11.658204 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:11.658174    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 17:27:11.658204 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:11.658201    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 17:27:11.658351 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:11.658215    2577 projected.go:196] Error preparing data for projected volume kube-api-access-fnfv6 for pod openshift-network-diagnostics/network-check-target-vm97k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 17:27:11.658351 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:11.658265    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6 podName:98fcc6f5-705d-415c-b08c-87dc618baca2 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:15.658246918 +0000 UTC m=+10.111810895 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fnfv6" (UniqueName: "kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6") pod "network-check-target-vm97k" (UID: "98fcc6f5-705d-415c-b08c-87dc618baca2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 17:27:11.658351 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:11.658333    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 17:27:11.658521 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:11.658367    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs podName:845cc472-f650-4adc-81b0-94b11e29ecfd nodeName:}" failed. No retries permitted until 2026-07-30 17:27:15.658354874 +0000 UTC m=+10.111918850 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs") pod "network-metrics-daemon-lwrkg" (UID: "845cc472-f650-4adc-81b0-94b11e29ecfd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 17:27:12.110616 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:12.110534    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:12.111125 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:12.110665    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vm97k" podUID="98fcc6f5-705d-415c-b08c-87dc618baca2"
Jul 30 17:27:13.104984 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:13.104945    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:13.105170 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:13.105077    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4vp7" podUID="a3251e0d-bc32-44a8-848d-2e64df31b7cc"
Jul 30 17:27:13.105552 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:13.105531    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:13.105686 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:13.105641    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwrkg" podUID="845cc472-f650-4adc-81b0-94b11e29ecfd"
Jul 30 17:27:14.105511 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:14.105031    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:14.105511 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:14.105164    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vm97k" podUID="98fcc6f5-705d-415c-b08c-87dc618baca2"
Jul 30 17:27:15.105158 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:15.105098    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:15.105354 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:15.105290    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4vp7" podUID="a3251e0d-bc32-44a8-848d-2e64df31b7cc"
Jul 30 17:27:15.105449 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:15.105400    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:15.105507 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:15.105486    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwrkg" podUID="845cc472-f650-4adc-81b0-94b11e29ecfd"
Jul 30 17:27:15.694231 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:15.694188    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnfv6\" (UniqueName: \"kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6\") pod \"network-check-target-vm97k\" (UID: \"98fcc6f5-705d-415c-b08c-87dc618baca2\") " pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:15.694723 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:15.694249    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs\") pod \"network-metrics-daemon-lwrkg\" (UID: \"845cc472-f650-4adc-81b0-94b11e29ecfd\") " pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:15.694723 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:15.694278    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret\") pod \"global-pull-secret-syncer-f4vp7\" (UID: \"a3251e0d-bc32-44a8-848d-2e64df31b7cc\") " pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:15.694723 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:15.694386    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 17:27:15.694723 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:15.694400    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 17:27:15.694723 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:15.694398    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 17:27:15.694723 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:15.694467    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs podName:845cc472-f650-4adc-81b0-94b11e29ecfd nodeName:}" failed. No retries permitted until 2026-07-30 17:27:23.694444512 +0000 UTC m=+18.148008486 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs") pod "network-metrics-daemon-lwrkg" (UID: "845cc472-f650-4adc-81b0-94b11e29ecfd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 17:27:15.694723 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:15.694468    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 17:27:15.694723 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:15.694491    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret podName:a3251e0d-bc32-44a8-848d-2e64df31b7cc nodeName:}" failed. No retries permitted until 2026-07-30 17:27:23.694477958 +0000 UTC m=+18.148041932 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret") pod "global-pull-secret-syncer-f4vp7" (UID: "a3251e0d-bc32-44a8-848d-2e64df31b7cc") : object "kube-system"/"original-pull-secret" not registered
Jul 30 17:27:15.694723 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:15.694500    2577 projected.go:196] Error preparing data for projected volume kube-api-access-fnfv6 for pod openshift-network-diagnostics/network-check-target-vm97k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 17:27:15.694723 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:15.694552    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6 podName:98fcc6f5-705d-415c-b08c-87dc618baca2 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:23.694531452 +0000 UTC m=+18.148095445 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fnfv6" (UniqueName: "kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6") pod "network-check-target-vm97k" (UID: "98fcc6f5-705d-415c-b08c-87dc618baca2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 17:27:16.108236 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:16.108155    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:16.108421 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:16.108279    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vm97k" podUID="98fcc6f5-705d-415c-b08c-87dc618baca2"
Jul 30 17:27:17.104417 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:17.104378    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:17.104928 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:17.104520    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4vp7" podUID="a3251e0d-bc32-44a8-848d-2e64df31b7cc"
Jul 30 17:27:17.105006 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:17.104948    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:17.105074 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:17.105061    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwrkg" podUID="845cc472-f650-4adc-81b0-94b11e29ecfd"
Jul 30 17:27:18.105450 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:18.104866    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:18.105450 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:18.105010    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vm97k" podUID="98fcc6f5-705d-415c-b08c-87dc618baca2"
Jul 30 17:27:19.104462 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:19.104421    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:19.104640 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:19.104427    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:19.104640 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:19.104529    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4vp7" podUID="a3251e0d-bc32-44a8-848d-2e64df31b7cc"
Jul 30 17:27:19.104640 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:19.104625    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwrkg" podUID="845cc472-f650-4adc-81b0-94b11e29ecfd"
Jul 30 17:27:20.104445 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:20.104412    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:20.104840 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:20.104525    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vm97k" podUID="98fcc6f5-705d-415c-b08c-87dc618baca2"
Jul 30 17:27:21.104592 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:21.104511    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:21.104592 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:21.104546    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:21.105095 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:21.104637    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4vp7" podUID="a3251e0d-bc32-44a8-848d-2e64df31b7cc"
Jul 30 17:27:21.105095 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:21.104786    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwrkg" podUID="845cc472-f650-4adc-81b0-94b11e29ecfd"
Jul 30 17:27:22.105152 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:22.105113    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:22.105596 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:22.105239    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vm97k" podUID="98fcc6f5-705d-415c-b08c-87dc618baca2"
Jul 30 17:27:23.105041 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:23.105007    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:23.105264 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:23.105140    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4vp7" podUID="a3251e0d-bc32-44a8-848d-2e64df31b7cc"
Jul 30 17:27:23.105264 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:23.105190    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:23.105654 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:23.105311    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwrkg" podUID="845cc472-f650-4adc-81b0-94b11e29ecfd"
Jul 30 17:27:23.756741 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:23.756706    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs\") pod \"network-metrics-daemon-lwrkg\" (UID: \"845cc472-f650-4adc-81b0-94b11e29ecfd\") " pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:23.757062 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:23.756757    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret\") pod \"global-pull-secret-syncer-f4vp7\" (UID: \"a3251e0d-bc32-44a8-848d-2e64df31b7cc\") " pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:23.757062 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:23.756829    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnfv6\" (UniqueName: \"kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6\") pod \"network-check-target-vm97k\" (UID: \"98fcc6f5-705d-415c-b08c-87dc618baca2\") " pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:23.757062 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:23.756897    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 17:27:23.757062 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:23.756948    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 17:27:23.757062 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:23.756964    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs podName:845cc472-f650-4adc-81b0-94b11e29ecfd nodeName:}" failed. No retries permitted until 2026-07-30 17:27:39.756947277 +0000 UTC m=+34.210511262 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs") pod "network-metrics-daemon-lwrkg" (UID: "845cc472-f650-4adc-81b0-94b11e29ecfd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 17:27:23.757062 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:23.756966    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 17:27:23.757062 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:23.756983    2577 projected.go:196] Error preparing data for projected volume kube-api-access-fnfv6 for pod openshift-network-diagnostics/network-check-target-vm97k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 17:27:23.757062 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:23.757031    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6 podName:98fcc6f5-705d-415c-b08c-87dc618baca2 nodeName:}" failed. No retries permitted until 2026-07-30 17:27:39.757014356 +0000 UTC m=+34.210578343 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fnfv6" (UniqueName: "kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6") pod "network-check-target-vm97k" (UID: "98fcc6f5-705d-415c-b08c-87dc618baca2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 17:27:23.757062 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:23.756897    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 17:27:23.757406 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:23.757071    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret podName:a3251e0d-bc32-44a8-848d-2e64df31b7cc nodeName:}" failed. No retries permitted until 2026-07-30 17:27:39.757059869 +0000 UTC m=+34.210623857 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret") pod "global-pull-secret-syncer-f4vp7" (UID: "a3251e0d-bc32-44a8-848d-2e64df31b7cc") : object "kube-system"/"original-pull-secret" not registered
Jul 30 17:27:24.104624 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:24.104546    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:24.104795 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:24.104693    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vm97k" podUID="98fcc6f5-705d-415c-b08c-87dc618baca2"
Jul 30 17:27:25.104531 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:25.104491    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:25.104531 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:25.104518    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:25.105007 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:25.104626    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwrkg" podUID="845cc472-f650-4adc-81b0-94b11e29ecfd"
Jul 30 17:27:25.105007 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:25.104780    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4vp7" podUID="a3251e0d-bc32-44a8-848d-2e64df31b7cc"
Jul 30 17:27:26.105224 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:26.105187    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:26.105679 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:26.105289    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vm97k" podUID="98fcc6f5-705d-415c-b08c-87dc618baca2"
Jul 30 17:27:27.104987 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:27.104958    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:27.104987 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:27.105005    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:27.105249 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:27.105117    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwrkg" podUID="845cc472-f650-4adc-81b0-94b11e29ecfd"
Jul 30 17:27:27.105249 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:27.105194    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4vp7" podUID="a3251e0d-bc32-44a8-848d-2e64df31b7cc"
Jul 30 17:27:28.104673 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:28.104640    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:28.104959 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:28.104792    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vm97k" podUID="98fcc6f5-705d-415c-b08c-87dc618baca2"
Jul 30 17:27:29.104647 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:29.104445    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:29.105292 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:29.104472    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:29.105292 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:29.104748    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4vp7" podUID="a3251e0d-bc32-44a8-848d-2e64df31b7cc"
Jul 30 17:27:29.105292 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:29.104800    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwrkg" podUID="845cc472-f650-4adc-81b0-94b11e29ecfd"
Jul 30 17:27:29.371466 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:29.371214    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6wttc" event={"ID":"8cdacb83-3cbb-4989-a0d1-9bee40686af3","Type":"ContainerStarted","Data":"82d0a69c49bee64611c29ed705fbdd9a50e5727d622205d07b18b6cc3e82b9f0"}
Jul 30 17:27:29.373012 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:29.372960    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mxb44" event={"ID":"d9f46378-294e-4eda-a3c4-08073cac32f2","Type":"ContainerStarted","Data":"b49e8593b70a743ec1347000a8f97e81b6b001673d0db679dc6d9efca636ac77"}
Jul 30 17:27:29.375333 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:29.374795    2577 generic.go:356] "Generic (PLEG): container finished" podID="c30771bb-82ee-4ecb-a478-3b34bd426bc9" containerID="ba5fe36dcccd19130704908f59a898abad0baaa536b4b785cc625631e3b27399" exitCode=0
Jul 30 17:27:29.375333 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:29.374850    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ctb9m" event={"ID":"c30771bb-82ee-4ecb-a478-3b34bd426bc9","Type":"ContainerDied","Data":"ba5fe36dcccd19130704908f59a898abad0baaa536b4b785cc625631e3b27399"}
Jul 30 17:27:29.376685 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:29.376666    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hkwdn" event={"ID":"8267208d-1805-4410-bf0c-595b1f1604f1","Type":"ContainerStarted","Data":"29f4e1d31d28c4b6d7e3a4ae203e1a964868e496d0da7b6d28b13c3f2e34f14b"}
Jul 30 17:27:29.378474 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:29.378393    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sfrfh" event={"ID":"7eb46cea-1c3f-4201-9b43-025d799ca911","Type":"ContainerStarted","Data":"5d1fcfd7669436fd559d31a0fa75c02605e549d8d60d72e53ebb9f8f52a40571"}
Jul 30 17:27:29.380021 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:29.379998    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm" event={"ID":"667a3ad7-25b2-430a-9878-94f8e7fa9069","Type":"ContainerStarted","Data":"bf6ddde2f8874e5a7edd0bed11eace721b329a61e97c439455ee6705f1ebadb7"}
Jul 30 17:27:29.383367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:29.383343    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l" event={"ID":"642de0d3-59f9-4601-bbc8-94df8e60e3e8","Type":"ContainerStarted","Data":"5ad2279eebd0add281fe43ccb30b98781bd31079527a3167137642b0678aaad4"}
Jul 30 17:27:29.383461 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:29.383407    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l" event={"ID":"642de0d3-59f9-4601-bbc8-94df8e60e3e8","Type":"ContainerStarted","Data":"f95541dcaf4dfe35bbaffefb2aa3fb9a0e067ff2240ea5305224235bb53611d8"}
Jul 30 17:27:29.383461 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:29.383436    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l" event={"ID":"642de0d3-59f9-4601-bbc8-94df8e60e3e8","Type":"ContainerStarted","Data":"b96a3fc37b3b5cd17f6f192725c29fc58eb653b32bb0061be2ff06a00a7f817f"}
Jul 30 17:27:29.383565 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:29.383471    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l" event={"ID":"642de0d3-59f9-4601-bbc8-94df8e60e3e8","Type":"ContainerStarted","Data":"6ad043ab0088c6585ec5704b5b115ee2b7469115e1ba4260df97abd83b769192"}
Jul 30 17:27:29.383565 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:29.383499    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l" event={"ID":"642de0d3-59f9-4601-bbc8-94df8e60e3e8","Type":"ContainerStarted","Data":"718a149b1b59fd086876ab2ac162b3736df22e84e397b94bfcb8565da2527914"}
Jul 30 17:27:29.383565 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:29.383527    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l" event={"ID":"642de0d3-59f9-4601-bbc8-94df8e60e3e8","Type":"ContainerStarted","Data":"5f108d68da63216b7eb97382be1c47dfded5ce62fb42a77c2211751b25cc24d4"}
Jul 30 17:27:29.385107 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:29.385086    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9nsjh" event={"ID":"0cb5d046-0bb1-452a-8562-c46ed626cf89","Type":"ContainerStarted","Data":"42039935ec94e87ca63d3f5bffb4ad9962a2fe5f705cbe249670c37ea0d9fc9f"}
Jul 30 17:27:29.386783 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:29.386742    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6wttc" podStartSLOduration=4.037535708 podStartE2EDuration="23.386725106s" totalImagesPullingTime="19.349189398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:27:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:29.386333831 +0000 UTC m=+23.839897826" watchObservedRunningTime="2026-07-30 17:27:29.386725106 +0000 UTC m=+23.840289099"
Jul 30 17:27:29.401427 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:29.401387    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hkwdn" podStartSLOduration=3.051478669 podStartE2EDuration="22.401372926s" totalImagesPullingTime="19.349894257s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:27:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:29.400704231 +0000 UTC m=+23.854268223" watchObservedRunningTime="2026-07-30 17:27:29.401372926 +0000 UTC m=+23.854936917"
Jul 30 17:27:29.415673 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:29.415639    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mxb44" podStartSLOduration=3.057094997 podStartE2EDuration="22.415627894s" totalImagesPullingTime="19.358532897s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:27:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:29.415301918 +0000 UTC m=+23.868865908" watchObservedRunningTime="2026-07-30 17:27:29.415627894 +0000 UTC m=+23.869191884"
Jul 30 17:27:29.456892 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:29.456808    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9nsjh" podStartSLOduration=3.05818519 podStartE2EDuration="22.456792544s" totalImagesPullingTime="19.398607354s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:27:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:29.43488533 +0000 UTC m=+23.888449325" watchObservedRunningTime="2026-07-30 17:27:29.456792544 +0000 UTC m=+23.910356536"
Jul 30 17:27:29.493838 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:29.493779    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:27:29.561424 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:29.561389    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:27:29.662072 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:29.662042    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:27:30.095173 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:30.095142    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 30 17:27:30.106661 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:30.104709    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:30.106661 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:30.104849    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vm97k" podUID="98fcc6f5-705d-415c-b08c-87dc618baca2"
Jul 30 17:27:30.389729 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:30.389686    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ppvt7" event={"ID":"8d8ce49e-b875-46ab-8175-7979aa8a1ce5","Type":"ContainerStarted","Data":"a754e7e75fd8163f15bb51818cbf53c61aa7daf82381ea7da024177d853a1608"}
Jul 30 17:27:30.392171 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:30.391784    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm" event={"ID":"667a3ad7-25b2-430a-9878-94f8e7fa9069","Type":"ContainerStarted","Data":"a6a86a226f39a5207394930d8e26022c9986e9cf3a69e3289ef9cff2973192e1"}
Jul 30 17:27:30.404689 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:30.404631    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-sfrfh" podStartSLOduration=4.048327178 podStartE2EDuration="23.4046096s" totalImagesPullingTime="19.356282422s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:27:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:29.474564108 +0000 UTC m=+23.928128110" watchObservedRunningTime="2026-07-30 17:27:30.4046096 +0000 UTC m=+24.858173594"
Jul 30 17:27:30.528631 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:30.528589    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:27:30.602524 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:30.602485    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:27:30.983077 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:30.982955    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-30T17:27:30.095163936Z","UUID":"2bdad0b6-73ca-425b-9232-5cb08648a284","Handler":null,"Name":"","Endpoint":""}
Jul 30 17:27:30.984702 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:30.984664    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 30 17:27:30.984842 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:30.984712    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 30 17:27:31.104633 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:31.104597    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:31.104633 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:31.104628    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:31.104895 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:31.104719    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4vp7" podUID="a3251e0d-bc32-44a8-848d-2e64df31b7cc"
Jul 30 17:27:31.104956 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:31.104927    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwrkg" podUID="845cc472-f650-4adc-81b0-94b11e29ecfd"
Jul 30 17:27:31.396991 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:31.396955    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm" event={"ID":"667a3ad7-25b2-430a-9878-94f8e7fa9069","Type":"ContainerStarted","Data":"95df94e84530987d7534b80cc0fe34fc068aa37795ed8601c901a29863af4e6c"}
Jul 30 17:27:31.401292 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:31.401261    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l" event={"ID":"642de0d3-59f9-4601-bbc8-94df8e60e3e8","Type":"ContainerStarted","Data":"492dd9d704f1f3b0997e6eb229491f66230481360824706c4e9e4a2f24b83344"}
Jul 30 17:27:31.414124 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:31.414070    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fk7lm" podStartSLOduration=3.870880857 podStartE2EDuration="25.414050647s" totalImagesPullingTime="21.54316979s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:27:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:31.413900967 +0000 UTC m=+25.867464959" watchObservedRunningTime="2026-07-30 17:27:31.414050647 +0000 UTC m=+25.867614639"
Jul 30 17:27:31.414701 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:31.414660    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ppvt7" podStartSLOduration=6.039741058 podStartE2EDuration="25.41465019s" totalImagesPullingTime="19.374909132s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:27:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:30.404139921 +0000 UTC m=+24.857703913" watchObservedRunningTime="2026-07-30 17:27:31.41465019 +0000 UTC m=+25.868214182"
Jul 30 17:27:31.582835 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:31.582786    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6wttc"
Jul 30 17:27:31.583567 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:31.583547    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6wttc"
Jul 30 17:27:32.104325 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:32.104292    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:32.104507 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:32.104422    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vm97k" podUID="98fcc6f5-705d-415c-b08c-87dc618baca2"
Jul 30 17:27:32.404049 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:32.403973    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6wttc"
Jul 30 17:27:32.404750 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:32.404447    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6wttc"
Jul 30 17:27:32.461739 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:32.461706    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:27:33.104260 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:33.104231    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:33.104436 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:33.104279    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:33.104436 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:33.104383    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwrkg" podUID="845cc472-f650-4adc-81b0-94b11e29ecfd"
Jul 30 17:27:33.104550 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:33.104522    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4vp7" podUID="a3251e0d-bc32-44a8-848d-2e64df31b7cc"
Jul 30 17:27:33.328220 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:33.328182    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:27:33.393194 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:33.393159    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:27:33.751179 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:33.751025    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:27:34.104597 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:34.104560    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:34.104755 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:34.104708    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vm97k" podUID="98fcc6f5-705d-415c-b08c-87dc618baca2"
Jul 30 17:27:34.409469 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:34.409430    2577 generic.go:356] "Generic (PLEG): container finished" podID="c30771bb-82ee-4ecb-a478-3b34bd426bc9" containerID="7b34e51ce31f5801f81d95d75a174654303e269e266164e3866a54758a22c819" exitCode=0
Jul 30 17:27:34.409668 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:34.409473    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ctb9m" event={"ID":"c30771bb-82ee-4ecb-a478-3b34bd426bc9","Type":"ContainerDied","Data":"7b34e51ce31f5801f81d95d75a174654303e269e266164e3866a54758a22c819"}
Jul 30 17:27:34.413232 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:34.413210    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l" event={"ID":"642de0d3-59f9-4601-bbc8-94df8e60e3e8","Type":"ContainerStarted","Data":"7b58042e201b86ad178071e09ff30a51fdaa86cb938a600f535ca57dcb493b73"}
Jul 30 17:27:34.462649 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:34.462602    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l" podStartSLOduration=9.019362011 podStartE2EDuration="28.462587924s" totalImagesPullingTime="19.443225913s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:27:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:34.462321152 +0000 UTC m=+28.915885144" watchObservedRunningTime="2026-07-30 17:27:34.462587924 +0000 UTC m=+28.916151954"
Jul 30 17:27:35.104451 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:35.104421    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:35.105116 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:35.104428    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:35.105116 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:35.104537    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4vp7" podUID="a3251e0d-bc32-44a8-848d-2e64df31b7cc"
Jul 30 17:27:35.105116 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:35.104688    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwrkg" podUID="845cc472-f650-4adc-81b0-94b11e29ecfd"
Jul 30 17:27:35.416096 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:35.415754    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:35.416247 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:35.416162    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:35.416247 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:35.416217    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:35.434595 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:35.434568    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:35.435355 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:35.435333    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:27:35.743033 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:35.742950    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vm97k"]
Jul 30 17:27:35.743197 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:35.743111    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:35.743262 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:35.743217    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vm97k" podUID="98fcc6f5-705d-415c-b08c-87dc618baca2"
Jul 30 17:27:35.746098 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:35.746063    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f4vp7"]
Jul 30 17:27:35.746241 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:35.746156    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:35.746241 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:35.746222    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4vp7" podUID="a3251e0d-bc32-44a8-848d-2e64df31b7cc"
Jul 30 17:27:35.746644 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:35.746622    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lwrkg"]
Jul 30 17:27:35.746741 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:35.746715    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:35.746845 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:35.746801    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwrkg" podUID="845cc472-f650-4adc-81b0-94b11e29ecfd"
Jul 30 17:27:36.419639 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:36.419608    2577 generic.go:356] "Generic (PLEG): container finished" podID="c30771bb-82ee-4ecb-a478-3b34bd426bc9" containerID="edba7b51b4ad90cf0848ce0e1577e7e1567aca31ad0d9c3821d0956306775195" exitCode=0
Jul 30 17:27:36.420097 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:36.419700    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ctb9m" event={"ID":"c30771bb-82ee-4ecb-a478-3b34bd426bc9","Type":"ContainerDied","Data":"edba7b51b4ad90cf0848ce0e1577e7e1567aca31ad0d9c3821d0956306775195"}
Jul 30 17:27:37.104537 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:37.104501    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:37.104696 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:37.104524    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:37.104696 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:37.104617    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:37.104696 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:37.104624    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4vp7" podUID="a3251e0d-bc32-44a8-848d-2e64df31b7cc"
Jul 30 17:27:37.104696 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:37.104683    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vm97k" podUID="98fcc6f5-705d-415c-b08c-87dc618baca2"
Jul 30 17:27:37.104894 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:37.104764    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwrkg" podUID="845cc472-f650-4adc-81b0-94b11e29ecfd"
Jul 30 17:27:37.936756 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:37.936723    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:27:38.007055 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:38.007015    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:27:38.256048 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:38.256012    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:27:38.327562 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:38.327478    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:27:38.426802 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:38.426766    2577 generic.go:356] "Generic (PLEG): container finished" podID="c30771bb-82ee-4ecb-a478-3b34bd426bc9" containerID="f6fed4f1cb0199f5a898b763303e9cf1029dcae767197565e90d866e278b2071" exitCode=0
Jul 30 17:27:38.426987 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:38.426841    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ctb9m" event={"ID":"c30771bb-82ee-4ecb-a478-3b34bd426bc9","Type":"ContainerDied","Data":"f6fed4f1cb0199f5a898b763303e9cf1029dcae767197565e90d866e278b2071"}
Jul 30 17:27:39.104361 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:39.104325    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:39.104783 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:39.104325    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:39.104783 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:39.104454    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4vp7" podUID="a3251e0d-bc32-44a8-848d-2e64df31b7cc"
Jul 30 17:27:39.104783 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:39.104332    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:39.104783 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:39.104560    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwrkg" podUID="845cc472-f650-4adc-81b0-94b11e29ecfd"
Jul 30 17:27:39.104783 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:39.104638    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vm97k" podUID="98fcc6f5-705d-415c-b08c-87dc618baca2"
Jul 30 17:27:39.770678 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:39.770641    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnfv6\" (UniqueName: \"kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6\") pod \"network-check-target-vm97k\" (UID: \"98fcc6f5-705d-415c-b08c-87dc618baca2\") " pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:39.770941 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:39.770704    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs\") pod \"network-metrics-daemon-lwrkg\" (UID: \"845cc472-f650-4adc-81b0-94b11e29ecfd\") " pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:39.770941 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:39.770730    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret\") pod \"global-pull-secret-syncer-f4vp7\" (UID: \"a3251e0d-bc32-44a8-848d-2e64df31b7cc\") " pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:39.770941 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:39.770874    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 17:27:39.770941 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:39.770939    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret podName:a3251e0d-bc32-44a8-848d-2e64df31b7cc nodeName:}" failed. No retries permitted until 2026-07-30 17:28:11.770920922 +0000 UTC m=+66.224484891 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret") pod "global-pull-secret-syncer-f4vp7" (UID: "a3251e0d-bc32-44a8-848d-2e64df31b7cc") : object "kube-system"/"original-pull-secret" not registered
Jul 30 17:27:39.771358 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:39.771339    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 17:27:39.771466 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:39.771362    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 17:27:39.771466 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:39.771374    2577 projected.go:196] Error preparing data for projected volume kube-api-access-fnfv6 for pod openshift-network-diagnostics/network-check-target-vm97k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 17:27:39.771466 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:39.771422    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6 podName:98fcc6f5-705d-415c-b08c-87dc618baca2 nodeName:}" failed. No retries permitted until 2026-07-30 17:28:11.771407325 +0000 UTC m=+66.224971309 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fnfv6" (UniqueName: "kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6") pod "network-check-target-vm97k" (UID: "98fcc6f5-705d-415c-b08c-87dc618baca2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 17:27:39.771623 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:39.771485    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 17:27:39.771623 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:39.771520    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs podName:845cc472-f650-4adc-81b0-94b11e29ecfd nodeName:}" failed. No retries permitted until 2026-07-30 17:28:11.771508207 +0000 UTC m=+66.225072179 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs") pod "network-metrics-daemon-lwrkg" (UID: "845cc472-f650-4adc-81b0-94b11e29ecfd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 17:27:41.104768 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.104726    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:41.104768 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.104779    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:41.105264 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.104804    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:41.105264 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:41.104919    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vm97k" podUID="98fcc6f5-705d-415c-b08c-87dc618baca2"
Jul 30 17:27:41.105264 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:41.105040    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f4vp7" podUID="a3251e0d-bc32-44a8-848d-2e64df31b7cc"
Jul 30 17:27:41.105264 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:41.105117    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lwrkg" podUID="845cc472-f650-4adc-81b0-94b11e29ecfd"
Jul 30 17:27:41.325686 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.325657    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-203.ec2.internal" event="NodeReady"
Jul 30 17:27:41.325883 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.325848    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 30 17:27:41.390030 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.389937    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5stgf"]
Jul 30 17:27:41.393094 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.393071    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5stgf"
Jul 30 17:27:41.395360 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.395335    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 30 17:27:41.395921 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.395898    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-brvhl"]
Jul 30 17:27:41.397274 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.396725    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 30 17:27:41.397274 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.397071    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tjl7j\""
Jul 30 17:27:41.398027 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.398003    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-brvhl"
Jul 30 17:27:41.400865 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.400732    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 30 17:27:41.400865 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.400790    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 30 17:27:41.401105 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.401083    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 30 17:27:41.401438 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.401303    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-sr7s9\""
Jul 30 17:27:41.407221 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.407196    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5stgf"]
Jul 30 17:27:41.418996 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.418961    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-brvhl"]
Jul 30 17:27:41.484120 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.484074    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9efc23b9-3f92-42ef-b797-7a8608be85cd-config-volume\") pod \"dns-default-5stgf\" (UID: \"9efc23b9-3f92-42ef-b797-7a8608be85cd\") " pod="openshift-dns/dns-default-5stgf"
Jul 30 17:27:41.484120 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.484111    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9efc23b9-3f92-42ef-b797-7a8608be85cd-tmp-dir\") pod \"dns-default-5stgf\" (UID: \"9efc23b9-3f92-42ef-b797-7a8608be85cd\") " pod="openshift-dns/dns-default-5stgf"
Jul 30 17:27:41.484339 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.484235    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9feb0d2f-7be2-4f8a-b867-289f917ca8a4-cert\") pod \"ingress-canary-brvhl\" (UID: \"9feb0d2f-7be2-4f8a-b867-289f917ca8a4\") " pod="openshift-ingress-canary/ingress-canary-brvhl"
Jul 30 17:27:41.484339 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.484279    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpcgp\" (UniqueName: \"kubernetes.io/projected/9efc23b9-3f92-42ef-b797-7a8608be85cd-kube-api-access-mpcgp\") pod \"dns-default-5stgf\" (UID: \"9efc23b9-3f92-42ef-b797-7a8608be85cd\") " pod="openshift-dns/dns-default-5stgf"
Jul 30 17:27:41.484339 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.484307    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsmmt\" (UniqueName: \"kubernetes.io/projected/9feb0d2f-7be2-4f8a-b867-289f917ca8a4-kube-api-access-jsmmt\") pod \"ingress-canary-brvhl\" (UID: \"9feb0d2f-7be2-4f8a-b867-289f917ca8a4\") " pod="openshift-ingress-canary/ingress-canary-brvhl"
Jul 30 17:27:41.484457 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.484352    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls\") pod \"dns-default-5stgf\" (UID: \"9efc23b9-3f92-42ef-b797-7a8608be85cd\") " pod="openshift-dns/dns-default-5stgf"
Jul 30 17:27:41.559729 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.559695    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:27:41.584964 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.584923    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9feb0d2f-7be2-4f8a-b867-289f917ca8a4-cert\") pod \"ingress-canary-brvhl\" (UID: \"9feb0d2f-7be2-4f8a-b867-289f917ca8a4\") " pod="openshift-ingress-canary/ingress-canary-brvhl"
Jul 30 17:27:41.584964 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.584966    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpcgp\" (UniqueName: \"kubernetes.io/projected/9efc23b9-3f92-42ef-b797-7a8608be85cd-kube-api-access-mpcgp\") pod \"dns-default-5stgf\" (UID: \"9efc23b9-3f92-42ef-b797-7a8608be85cd\") " pod="openshift-dns/dns-default-5stgf"
Jul 30 17:27:41.585173 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.584989    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsmmt\" (UniqueName: \"kubernetes.io/projected/9feb0d2f-7be2-4f8a-b867-289f917ca8a4-kube-api-access-jsmmt\") pod \"ingress-canary-brvhl\" (UID: \"9feb0d2f-7be2-4f8a-b867-289f917ca8a4\") " pod="openshift-ingress-canary/ingress-canary-brvhl"
Jul 30 17:27:41.585173 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.585023    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls\") pod \"dns-default-5stgf\" (UID: \"9efc23b9-3f92-42ef-b797-7a8608be85cd\") " pod="openshift-dns/dns-default-5stgf"
Jul 30 17:27:41.585173 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.585059    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9efc23b9-3f92-42ef-b797-7a8608be85cd-config-volume\") pod \"dns-default-5stgf\" (UID: \"9efc23b9-3f92-42ef-b797-7a8608be85cd\") " pod="openshift-dns/dns-default-5stgf"
Jul 30 17:27:41.585173 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.585074    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9efc23b9-3f92-42ef-b797-7a8608be85cd-tmp-dir\") pod \"dns-default-5stgf\" (UID: \"9efc23b9-3f92-42ef-b797-7a8608be85cd\") " pod="openshift-dns/dns-default-5stgf"
Jul 30 17:27:41.585359 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:41.585173    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 17:27:41.585359 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:41.585246    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls podName:9efc23b9-3f92-42ef-b797-7a8608be85cd nodeName:}" failed. No retries permitted until 2026-07-30 17:27:42.085222355 +0000 UTC m=+36.538786327 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls") pod "dns-default-5stgf" (UID: "9efc23b9-3f92-42ef-b797-7a8608be85cd") : secret "dns-default-metrics-tls" not found
Jul 30 17:27:41.585645 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.585626    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9efc23b9-3f92-42ef-b797-7a8608be85cd-tmp-dir\") pod \"dns-default-5stgf\" (UID: \"9efc23b9-3f92-42ef-b797-7a8608be85cd\") " pod="openshift-dns/dns-default-5stgf"
Jul 30 17:27:41.586054 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.586026    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9efc23b9-3f92-42ef-b797-7a8608be85cd-config-volume\") pod \"dns-default-5stgf\" (UID: \"9efc23b9-3f92-42ef-b797-7a8608be85cd\") " pod="openshift-dns/dns-default-5stgf"
Jul 30 17:27:41.590112 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.590090    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9feb0d2f-7be2-4f8a-b867-289f917ca8a4-cert\") pod \"ingress-canary-brvhl\" (UID: \"9feb0d2f-7be2-4f8a-b867-289f917ca8a4\") " pod="openshift-ingress-canary/ingress-canary-brvhl"
Jul 30 17:27:41.594698 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.594668    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpcgp\" (UniqueName: \"kubernetes.io/projected/9efc23b9-3f92-42ef-b797-7a8608be85cd-kube-api-access-mpcgp\") pod \"dns-default-5stgf\" (UID: \"9efc23b9-3f92-42ef-b797-7a8608be85cd\") " pod="openshift-dns/dns-default-5stgf"
Jul 30 17:27:41.594903 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.594877    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsmmt\" (UniqueName: \"kubernetes.io/projected/9feb0d2f-7be2-4f8a-b867-289f917ca8a4-kube-api-access-jsmmt\") pod \"ingress-canary-brvhl\" (UID: \"9feb0d2f-7be2-4f8a-b867-289f917ca8a4\") " pod="openshift-ingress-canary/ingress-canary-brvhl"
Jul 30 17:27:41.663797 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.663712    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:27:41.715922 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:41.715889    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-brvhl"
Jul 30 17:27:42.089040 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:42.088996    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls\") pod \"dns-default-5stgf\" (UID: \"9efc23b9-3f92-42ef-b797-7a8608be85cd\") " pod="openshift-dns/dns-default-5stgf"
Jul 30 17:27:42.089206 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:42.089153    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 17:27:42.089256 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:42.089237    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls podName:9efc23b9-3f92-42ef-b797-7a8608be85cd nodeName:}" failed. No retries permitted until 2026-07-30 17:27:43.089219072 +0000 UTC m=+37.542783063 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls") pod "dns-default-5stgf" (UID: "9efc23b9-3f92-42ef-b797-7a8608be85cd") : secret "dns-default-metrics-tls" not found
Jul 30 17:27:42.558590 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:42.558555    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-brvhl"]
Jul 30 17:27:42.570970 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:27:42.570776    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9feb0d2f_7be2_4f8a_b867_289f917ca8a4.slice/crio-ee23bdd0901910a28afd4fed78ff502815e03b0f3fd0c8458a39df01f96d4b39 WatchSource:0}: Error finding container ee23bdd0901910a28afd4fed78ff502815e03b0f3fd0c8458a39df01f96d4b39: Status 404 returned error can't find the container with id ee23bdd0901910a28afd4fed78ff502815e03b0f3fd0c8458a39df01f96d4b39
Jul 30 17:27:43.098221 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:43.098179    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls\") pod \"dns-default-5stgf\" (UID: \"9efc23b9-3f92-42ef-b797-7a8608be85cd\") " pod="openshift-dns/dns-default-5stgf"
Jul 30 17:27:43.098403 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:43.098314    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 17:27:43.098403 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:43.098384    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls podName:9efc23b9-3f92-42ef-b797-7a8608be85cd nodeName:}" failed. No retries permitted until 2026-07-30 17:27:45.098367495 +0000 UTC m=+39.551931467 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls") pod "dns-default-5stgf" (UID: "9efc23b9-3f92-42ef-b797-7a8608be85cd") : secret "dns-default-metrics-tls" not found
Jul 30 17:27:43.105002 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:43.104971    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:27:43.105151 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:43.105008    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:27:43.105151 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:43.104986    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:27:43.109597 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:43.109300    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 17:27:43.109597 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:43.109433    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 17:27:43.109597 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:43.109303    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7qqrb\""
Jul 30 17:27:43.109597 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:43.109303    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 17:27:43.109597 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:43.109524    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 17:27:43.109597 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:43.109461    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fjwv8\""
Jul 30 17:27:43.442258 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:43.442174    2577 generic.go:356] "Generic (PLEG): container finished" podID="c30771bb-82ee-4ecb-a478-3b34bd426bc9" containerID="4bce8d60da72e5f3e566c7b7ba628c6b945d5ff48856c714ec0179913e4c65b1" exitCode=0
Jul 30 17:27:43.442456 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:43.442265    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ctb9m" event={"ID":"c30771bb-82ee-4ecb-a478-3b34bd426bc9","Type":"ContainerDied","Data":"4bce8d60da72e5f3e566c7b7ba628c6b945d5ff48856c714ec0179913e4c65b1"}
Jul 30 17:27:43.443621 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:43.443588    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-brvhl" event={"ID":"9feb0d2f-7be2-4f8a-b867-289f917ca8a4","Type":"ContainerStarted","Data":"ee23bdd0901910a28afd4fed78ff502815e03b0f3fd0c8458a39df01f96d4b39"}
Jul 30 17:27:44.450056 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:44.450011    2577 generic.go:356] "Generic (PLEG): container finished" podID="c30771bb-82ee-4ecb-a478-3b34bd426bc9" containerID="7b05c85cc1e0f6ef604902353ea9039ebbd18e3e7bd55b004cec8c31374240e7" exitCode=0
Jul 30 17:27:44.450474 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:44.450078    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ctb9m" event={"ID":"c30771bb-82ee-4ecb-a478-3b34bd426bc9","Type":"ContainerDied","Data":"7b05c85cc1e0f6ef604902353ea9039ebbd18e3e7bd55b004cec8c31374240e7"}
Jul 30 17:27:45.113847 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:45.113790    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls\") pod \"dns-default-5stgf\" (UID: \"9efc23b9-3f92-42ef-b797-7a8608be85cd\") " pod="openshift-dns/dns-default-5stgf"
Jul 30 17:27:45.114567 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:45.114139    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 17:27:45.114567 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:45.114231    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls podName:9efc23b9-3f92-42ef-b797-7a8608be85cd nodeName:}" failed. No retries permitted until 2026-07-30 17:27:49.114206075 +0000 UTC m=+43.567770067 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls") pod "dns-default-5stgf" (UID: "9efc23b9-3f92-42ef-b797-7a8608be85cd") : secret "dns-default-metrics-tls" not found
Jul 30 17:27:45.456235 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:45.456158    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ctb9m" event={"ID":"c30771bb-82ee-4ecb-a478-3b34bd426bc9","Type":"ContainerStarted","Data":"9aeffe6f6e19e94be6bc24b5b1c155706d24e1d4ffa21fa6e7e351f39eac308b"}
Jul 30 17:27:45.457933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:45.457908    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-brvhl" event={"ID":"9feb0d2f-7be2-4f8a-b867-289f917ca8a4","Type":"ContainerStarted","Data":"65111a7f622ed5dfc83780f0d184563a8766e26bb508da3cd28a34814645d9c8"}
Jul 30 17:27:45.511551 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:45.511496    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ctb9m" podStartSLOduration=8.631967336 podStartE2EDuration="38.511474271s" totalImagesPullingTime="29.879506935s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:27:07 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:45.486941974 +0000 UTC m=+39.940505967" watchObservedRunningTime="2026-07-30 17:27:45.511474271 +0000 UTC m=+39.965038264"
Jul 30 17:27:45.661591 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:45.661418    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:27:45.760420 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:45.760389    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:27:45.844586 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:45.844560    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:27:46.125583 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:46.125490    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:27:46.200613 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:46.200589    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:27:46.727721 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:46.727679    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:27:49.141168 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:49.141124    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls\") pod \"dns-default-5stgf\" (UID: \"9efc23b9-3f92-42ef-b797-7a8608be85cd\") " pod="openshift-dns/dns-default-5stgf"
Jul 30 17:27:49.141540 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:49.141240    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 17:27:49.141540 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:49.141294    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls podName:9efc23b9-3f92-42ef-b797-7a8608be85cd nodeName:}" failed. No retries permitted until 2026-07-30 17:27:57.141279835 +0000 UTC m=+51.594843804 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls") pod "dns-default-5stgf" (UID: "9efc23b9-3f92-42ef-b797-7a8608be85cd") : secret "dns-default-metrics-tls" not found
Jul 30 17:27:57.196835 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:27:57.196776    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls\") pod \"dns-default-5stgf\" (UID: \"9efc23b9-3f92-42ef-b797-7a8608be85cd\") " pod="openshift-dns/dns-default-5stgf"
Jul 30 17:27:57.197334 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:57.196931    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 17:27:57.197334 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:27:57.197004    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls podName:9efc23b9-3f92-42ef-b797-7a8608be85cd nodeName:}" failed. No retries permitted until 2026-07-30 17:28:13.196984451 +0000 UTC m=+67.650548424 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls") pod "dns-default-5stgf" (UID: "9efc23b9-3f92-42ef-b797-7a8608be85cd") : secret "dns-default-metrics-tls" not found
Jul 30 17:28:00.247544 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:00.247507    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:28:00.332232 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:00.332204    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:28:04.445950 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.445903    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-brvhl" podStartSLOduration=21.038863784 podStartE2EDuration="23.445884305s" totalImagesPullingTime="2.407020521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:27:45.511745796 +0000 UTC m=+39.965309777" watchObservedRunningTime="2026-07-30 17:28:04.445884305 +0000 UTC m=+58.899448322"
Jul 30 17:28:04.446342 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.446069    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"]
Jul 30 17:28:04.448055 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.448040    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"
Jul 30 17:28:04.450897 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.450873    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 30 17:28:04.451774 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.451755    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 30 17:28:04.451921 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.451887    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 30 17:28:04.451921 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.451903    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 30 17:28:04.451921 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.451910    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 30 17:28:04.452140 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.451887    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 30 17:28:04.452679 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.452665    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 30 17:28:04.468962 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.468934    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"]
Jul 30 17:28:04.543860 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.543803    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f927e85-aa58-4a84-a5d2-038558a187f4-additional-service-ca\") pod \"cluster-proxy-proxy-agent-d74fb7447-vm2cp\" (UID: \"6f927e85-aa58-4a84-a5d2-038558a187f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"
Jul 30 17:28:04.544031 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.543908    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6f927e85-aa58-4a84-a5d2-038558a187f4-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-d74fb7447-vm2cp\" (UID: \"6f927e85-aa58-4a84-a5d2-038558a187f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"
Jul 30 17:28:04.544031 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.543953    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krdfh\" (UniqueName: \"kubernetes.io/projected/6f927e85-aa58-4a84-a5d2-038558a187f4-kube-api-access-krdfh\") pod \"cluster-proxy-proxy-agent-d74fb7447-vm2cp\" (UID: \"6f927e85-aa58-4a84-a5d2-038558a187f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"
Jul 30 17:28:04.544031 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.543974    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6f927e85-aa58-4a84-a5d2-038558a187f4-hub\") pod \"cluster-proxy-proxy-agent-d74fb7447-vm2cp\" (UID: \"6f927e85-aa58-4a84-a5d2-038558a187f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"
Jul 30 17:28:04.544031 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.543992    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6f927e85-aa58-4a84-a5d2-038558a187f4-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-d74fb7447-vm2cp\" (UID: \"6f927e85-aa58-4a84-a5d2-038558a187f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"
Jul 30 17:28:04.544031 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.544022    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6f927e85-aa58-4a84-a5d2-038558a187f4-ca\") pod \"cluster-proxy-proxy-agent-d74fb7447-vm2cp\" (UID: \"6f927e85-aa58-4a84-a5d2-038558a187f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"
Jul 30 17:28:04.645160 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.645130    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6f927e85-aa58-4a84-a5d2-038558a187f4-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-d74fb7447-vm2cp\" (UID: \"6f927e85-aa58-4a84-a5d2-038558a187f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"
Jul 30 17:28:04.645307 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.645188    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krdfh\" (UniqueName: \"kubernetes.io/projected/6f927e85-aa58-4a84-a5d2-038558a187f4-kube-api-access-krdfh\") pod \"cluster-proxy-proxy-agent-d74fb7447-vm2cp\" (UID: \"6f927e85-aa58-4a84-a5d2-038558a187f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"
Jul 30 17:28:04.645307 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.645214    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6f927e85-aa58-4a84-a5d2-038558a187f4-hub\") pod \"cluster-proxy-proxy-agent-d74fb7447-vm2cp\" (UID: \"6f927e85-aa58-4a84-a5d2-038558a187f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"
Jul 30 17:28:04.645307 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.645240    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6f927e85-aa58-4a84-a5d2-038558a187f4-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-d74fb7447-vm2cp\" (UID: \"6f927e85-aa58-4a84-a5d2-038558a187f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"
Jul 30 17:28:04.645307 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.645266    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6f927e85-aa58-4a84-a5d2-038558a187f4-ca\") pod \"cluster-proxy-proxy-agent-d74fb7447-vm2cp\" (UID: \"6f927e85-aa58-4a84-a5d2-038558a187f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"
Jul 30 17:28:04.645442 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.645314    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f927e85-aa58-4a84-a5d2-038558a187f4-additional-service-ca\") pod \"cluster-proxy-proxy-agent-d74fb7447-vm2cp\" (UID: \"6f927e85-aa58-4a84-a5d2-038558a187f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"
Jul 30 17:28:04.646479 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.646449    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f927e85-aa58-4a84-a5d2-038558a187f4-additional-service-ca\") pod \"cluster-proxy-proxy-agent-d74fb7447-vm2cp\" (UID: \"6f927e85-aa58-4a84-a5d2-038558a187f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"
Jul 30 17:28:04.651142 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.651042    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6f927e85-aa58-4a84-a5d2-038558a187f4-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-d74fb7447-vm2cp\" (UID: \"6f927e85-aa58-4a84-a5d2-038558a187f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"
Jul 30 17:28:04.651142 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.651060    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6f927e85-aa58-4a84-a5d2-038558a187f4-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-d74fb7447-vm2cp\" (UID: \"6f927e85-aa58-4a84-a5d2-038558a187f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"
Jul 30 17:28:04.651142 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.651046    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6f927e85-aa58-4a84-a5d2-038558a187f4-hub\") pod \"cluster-proxy-proxy-agent-d74fb7447-vm2cp\" (UID: \"6f927e85-aa58-4a84-a5d2-038558a187f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"
Jul 30 17:28:04.651751 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.651731    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6f927e85-aa58-4a84-a5d2-038558a187f4-ca\") pod \"cluster-proxy-proxy-agent-d74fb7447-vm2cp\" (UID: \"6f927e85-aa58-4a84-a5d2-038558a187f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"
Jul 30 17:28:04.655300 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.655278    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krdfh\" (UniqueName: \"kubernetes.io/projected/6f927e85-aa58-4a84-a5d2-038558a187f4-kube-api-access-krdfh\") pod \"cluster-proxy-proxy-agent-d74fb7447-vm2cp\" (UID: \"6f927e85-aa58-4a84-a5d2-038558a187f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"
Jul 30 17:28:04.765952 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.765915    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"
Jul 30 17:28:04.902952 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:04.902922    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"]
Jul 30 17:28:04.906106 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:28:04.906075    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f927e85_aa58_4a84_a5d2_038558a187f4.slice/crio-4a9bba15e7862e0a96cc08c3c472bae5b797472f3bf7ea87dca4320126b6f71a WatchSource:0}: Error finding container 4a9bba15e7862e0a96cc08c3c472bae5b797472f3bf7ea87dca4320126b6f71a: Status 404 returned error can't find the container with id 4a9bba15e7862e0a96cc08c3c472bae5b797472f3bf7ea87dca4320126b6f71a
Jul 30 17:28:05.504989 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:05.504946    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp" event={"ID":"6f927e85-aa58-4a84-a5d2-038558a187f4","Type":"ContainerStarted","Data":"4a9bba15e7862e0a96cc08c3c472bae5b797472f3bf7ea87dca4320126b6f71a"}
Jul 30 17:28:07.435634 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:07.435599    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b6r8l"
Jul 30 17:28:07.567072 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:07.566962    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:28:07.667484 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:07.667441    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:28:07.782505 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:07.782474    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:28:09.516633 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:09.516598    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp" event={"ID":"6f927e85-aa58-4a84-a5d2-038558a187f4","Type":"ContainerStarted","Data":"3871c7973ef6fe60ce06e67c72d4fbfeb9114c084f2dd7ff477b2278360a0226"}
Jul 30 17:28:09.516633 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:09.516641    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp" event={"ID":"6f927e85-aa58-4a84-a5d2-038558a187f4","Type":"ContainerStarted","Data":"4020dd01fbdf0347f9244b4bf6e27d5daf4215707e24a72e291fae282f27c233"}
Jul 30 17:28:09.517080 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:09.516653    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp" event={"ID":"6f927e85-aa58-4a84-a5d2-038558a187f4","Type":"ContainerStarted","Data":"e1cf72c2d25a4617481a20e2d075d1e9bb26a75833fa0261cb59e436335dccb0"}
Jul 30 17:28:09.537526 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:09.537472    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp" podStartSLOduration=1.567151291 podStartE2EDuration="5.537456499s" totalImagesPullingTime="3.970305208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:28:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:28:09.536745424 +0000 UTC m=+63.990309415" watchObservedRunningTime="2026-07-30 17:28:09.537456499 +0000 UTC m=+63.991020525"
Jul 30 17:28:11.795159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:11.795122    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnfv6\" (UniqueName: \"kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6\") pod \"network-check-target-vm97k\" (UID: \"98fcc6f5-705d-415c-b08c-87dc618baca2\") " pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:28:11.795159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:11.795161    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs\") pod \"network-metrics-daemon-lwrkg\" (UID: \"845cc472-f650-4adc-81b0-94b11e29ecfd\") " pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:28:11.795582 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:11.795182    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret\") pod \"global-pull-secret-syncer-f4vp7\" (UID: \"a3251e0d-bc32-44a8-848d-2e64df31b7cc\") " pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:28:11.798077 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:11.798057    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 17:28:11.798148 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:11.798062    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 17:28:11.798245 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:11.798228    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 17:28:11.806177 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:28:11.806155    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 30 17:28:11.806312 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:28:11.806233    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs podName:845cc472-f650-4adc-81b0-94b11e29ecfd nodeName:}" failed. No retries permitted until 2026-07-30 17:29:15.806216988 +0000 UTC m=+130.259780958 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs") pod "network-metrics-daemon-lwrkg" (UID: "845cc472-f650-4adc-81b0-94b11e29ecfd") : secret "metrics-daemon-secret" not found
Jul 30 17:28:11.808378 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:11.808357    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3251e0d-bc32-44a8-848d-2e64df31b7cc-original-pull-secret\") pod \"global-pull-secret-syncer-f4vp7\" (UID: \"a3251e0d-bc32-44a8-848d-2e64df31b7cc\") " pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:28:11.808536 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:11.808522    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 17:28:11.818663 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:11.818636    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnfv6\" (UniqueName: \"kubernetes.io/projected/98fcc6f5-705d-415c-b08c-87dc618baca2-kube-api-access-fnfv6\") pod \"network-check-target-vm97k\" (UID: \"98fcc6f5-705d-415c-b08c-87dc618baca2\") " pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:28:11.917170 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:11.917127    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f4vp7"
Jul 30 17:28:11.933241 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:11.933217    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7qqrb\""
Jul 30 17:28:11.941357 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:11.941335    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:28:12.083304 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:12.083270    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f4vp7"]
Jul 30 17:28:12.086342 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:28:12.086309    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3251e0d_bc32_44a8_848d_2e64df31b7cc.slice/crio-5a0eec518e79ffea0ebbfea5a4fea4bdc823b344c1ebbde92803deb219b48517 WatchSource:0}: Error finding container 5a0eec518e79ffea0ebbfea5a4fea4bdc823b344c1ebbde92803deb219b48517: Status 404 returned error can't find the container with id 5a0eec518e79ffea0ebbfea5a4fea4bdc823b344c1ebbde92803deb219b48517
Jul 30 17:28:12.098013 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:12.097984    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vm97k"]
Jul 30 17:28:12.103138 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:28:12.103116    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98fcc6f5_705d_415c_b08c_87dc618baca2.slice/crio-98a59d05261c9ce9d2342ca2744f3bed5a1bb36e83883864b47bef18ee10ceb4 WatchSource:0}: Error finding container 98a59d05261c9ce9d2342ca2744f3bed5a1bb36e83883864b47bef18ee10ceb4: Status 404 returned error can't find the container with id 98a59d05261c9ce9d2342ca2744f3bed5a1bb36e83883864b47bef18ee10ceb4
Jul 30 17:28:12.525397 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:12.525356    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vm97k" event={"ID":"98fcc6f5-705d-415c-b08c-87dc618baca2","Type":"ContainerStarted","Data":"98a59d05261c9ce9d2342ca2744f3bed5a1bb36e83883864b47bef18ee10ceb4"}
Jul 30 17:28:12.526361 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:12.526338    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f4vp7" event={"ID":"a3251e0d-bc32-44a8-848d-2e64df31b7cc","Type":"ContainerStarted","Data":"5a0eec518e79ffea0ebbfea5a4fea4bdc823b344c1ebbde92803deb219b48517"}
Jul 30 17:28:13.211344 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:13.210720    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls\") pod \"dns-default-5stgf\" (UID: \"9efc23b9-3f92-42ef-b797-7a8608be85cd\") " pod="openshift-dns/dns-default-5stgf"
Jul 30 17:28:13.211344 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:28:13.210937    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 17:28:13.211344 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:28:13.211007    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls podName:9efc23b9-3f92-42ef-b797-7a8608be85cd nodeName:}" failed. No retries permitted until 2026-07-30 17:28:45.210987556 +0000 UTC m=+99.664551527 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls") pod "dns-default-5stgf" (UID: "9efc23b9-3f92-42ef-b797-7a8608be85cd") : secret "dns-default-metrics-tls" not found
Jul 30 17:28:17.540717 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:17.540672    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vm97k" event={"ID":"98fcc6f5-705d-415c-b08c-87dc618baca2","Type":"ContainerStarted","Data":"b752063dbe82d3883e2c834d338fed4acf61addb6ff17224b68241bd903e6452"}
Jul 30 17:28:17.541238 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:17.540767    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:28:17.542193 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:17.542172    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f4vp7" event={"ID":"a3251e0d-bc32-44a8-848d-2e64df31b7cc","Type":"ContainerStarted","Data":"d90a7fb5c2c33537a4429786ed26309ccd1a9d5fb3fa186ca112450d4dbcb075"}
Jul 30 17:28:17.556560 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:17.556409    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vm97k" podStartSLOduration=67.097117173 podStartE2EDuration="1m11.556393479s" totalImagesPullingTime="4.459276306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:27:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:28:17.555748934 +0000 UTC m=+72.009312926" watchObservedRunningTime="2026-07-30 17:28:17.556393479 +0000 UTC m=+72.009957471"
Jul 30 17:28:17.575383 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:17.575325    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-f4vp7" podStartSLOduration=66.094326522 podStartE2EDuration="1m10.575308611s" totalImagesPullingTime="4.480982089s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:27:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:28:17.574683854 +0000 UTC m=+72.028247844" watchObservedRunningTime="2026-07-30 17:28:17.575308611 +0000 UTC m=+72.028872604"
Jul 30 17:28:45.232347 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:45.232321    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls\") pod \"dns-default-5stgf\" (UID: \"9efc23b9-3f92-42ef-b797-7a8608be85cd\") " pod="openshift-dns/dns-default-5stgf"
Jul 30 17:28:45.232662 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:28:45.232442    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 17:28:45.232662 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:28:45.232502    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls podName:9efc23b9-3f92-42ef-b797-7a8608be85cd nodeName:}" failed. No retries permitted until 2026-07-30 17:29:49.23248837 +0000 UTC m=+163.686052340 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls") pod "dns-default-5stgf" (UID: "9efc23b9-3f92-42ef-b797-7a8608be85cd") : secret "dns-default-metrics-tls" not found
Jul 30 17:28:48.547001 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:28:48.546969    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vm97k"
Jul 30 17:29:15.838120 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:29:15.838079    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs\") pod \"network-metrics-daemon-lwrkg\" (UID: \"845cc472-f650-4adc-81b0-94b11e29ecfd\") " pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:29:15.838529 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:29:15.838224    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 30 17:29:15.838529 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:29:15.838286    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs podName:845cc472-f650-4adc-81b0-94b11e29ecfd nodeName:}" failed. No retries permitted until 2026-07-30 17:31:17.83827151 +0000 UTC m=+252.291835479 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs") pod "network-metrics-daemon-lwrkg" (UID: "845cc472-f650-4adc-81b0-94b11e29ecfd") : secret "metrics-daemon-secret" not found
Jul 30 17:29:44.408117 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:29:44.408070    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5stgf" podUID="9efc23b9-3f92-42ef-b797-7a8608be85cd"
Jul 30 17:29:44.788621 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:29:44.788592    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5stgf"
Jul 30 17:29:46.125839 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:29:46.125781    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-lwrkg" podUID="845cc472-f650-4adc-81b0-94b11e29ecfd"
Jul 30 17:29:49.271070 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:29:49.271023    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls\") pod \"dns-default-5stgf\" (UID: \"9efc23b9-3f92-42ef-b797-7a8608be85cd\") " pod="openshift-dns/dns-default-5stgf"
Jul 30 17:29:49.271451 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:29:49.271190    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 17:29:49.271451 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:29:49.271256    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls podName:9efc23b9-3f92-42ef-b797-7a8608be85cd nodeName:}" failed. No retries permitted until 2026-07-30 17:31:51.271235969 +0000 UTC m=+285.724799945 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls") pod "dns-default-5stgf" (UID: "9efc23b9-3f92-42ef-b797-7a8608be85cd") : secret "dns-default-metrics-tls" not found
Jul 30 17:30:00.104374 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:30:00.104293    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:30:37.096490 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:30:37.096459    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:12.415269 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.415235    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-9pdgh"]
Jul 30 17:31:12.417021 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.417006    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-9pdgh"
Jul 30 17:31:12.419240 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.419210    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 30 17:31:12.419363 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.419293    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 30 17:31:12.419455 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.419443    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-p7t46\""
Jul 30 17:31:12.420370 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.420347    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 30 17:31:12.420468 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.420397    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 30 17:31:12.425352 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.425336    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 30 17:31:12.432237 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.432217    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-9pdgh"]
Jul 30 17:31:12.516740 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.516701    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7419876-6314-486e-b064-ee186d6cac15-serving-cert\") pod \"insights-operator-5885fdb759-9pdgh\" (UID: \"a7419876-6314-486e-b064-ee186d6cac15\") " pod="openshift-insights/insights-operator-5885fdb759-9pdgh"
Jul 30 17:31:12.516945 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.516756    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a7419876-6314-486e-b064-ee186d6cac15-snapshots\") pod \"insights-operator-5885fdb759-9pdgh\" (UID: \"a7419876-6314-486e-b064-ee186d6cac15\") " pod="openshift-insights/insights-operator-5885fdb759-9pdgh"
Jul 30 17:31:12.516945 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.516778    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7419876-6314-486e-b064-ee186d6cac15-tmp\") pod \"insights-operator-5885fdb759-9pdgh\" (UID: \"a7419876-6314-486e-b064-ee186d6cac15\") " pod="openshift-insights/insights-operator-5885fdb759-9pdgh"
Jul 30 17:31:12.516945 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.516797    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7419876-6314-486e-b064-ee186d6cac15-service-ca-bundle\") pod \"insights-operator-5885fdb759-9pdgh\" (UID: \"a7419876-6314-486e-b064-ee186d6cac15\") " pod="openshift-insights/insights-operator-5885fdb759-9pdgh"
Jul 30 17:31:12.516945 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.516891    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7419876-6314-486e-b064-ee186d6cac15-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-9pdgh\" (UID: \"a7419876-6314-486e-b064-ee186d6cac15\") " pod="openshift-insights/insights-operator-5885fdb759-9pdgh"
Jul 30 17:31:12.517205 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.516948    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8wh4\" (UniqueName: \"kubernetes.io/projected/a7419876-6314-486e-b064-ee186d6cac15-kube-api-access-p8wh4\") pod \"insights-operator-5885fdb759-9pdgh\" (UID: \"a7419876-6314-486e-b064-ee186d6cac15\") " pod="openshift-insights/insights-operator-5885fdb759-9pdgh"
Jul 30 17:31:12.579456 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.579429    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:12.617557 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.617505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a7419876-6314-486e-b064-ee186d6cac15-snapshots\") pod \"insights-operator-5885fdb759-9pdgh\" (UID: \"a7419876-6314-486e-b064-ee186d6cac15\") " pod="openshift-insights/insights-operator-5885fdb759-9pdgh"
Jul 30 17:31:12.617557 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.617561    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7419876-6314-486e-b064-ee186d6cac15-tmp\") pod \"insights-operator-5885fdb759-9pdgh\" (UID: \"a7419876-6314-486e-b064-ee186d6cac15\") " pod="openshift-insights/insights-operator-5885fdb759-9pdgh"
Jul 30 17:31:12.617752 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.617606    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7419876-6314-486e-b064-ee186d6cac15-service-ca-bundle\") pod \"insights-operator-5885fdb759-9pdgh\" (UID: \"a7419876-6314-486e-b064-ee186d6cac15\") " pod="openshift-insights/insights-operator-5885fdb759-9pdgh"
Jul 30 17:31:12.617752 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.617647    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7419876-6314-486e-b064-ee186d6cac15-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-9pdgh\" (UID: \"a7419876-6314-486e-b064-ee186d6cac15\") " pod="openshift-insights/insights-operator-5885fdb759-9pdgh"
Jul 30 17:31:12.617752 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.617699    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8wh4\" (UniqueName: \"kubernetes.io/projected/a7419876-6314-486e-b064-ee186d6cac15-kube-api-access-p8wh4\") pod \"insights-operator-5885fdb759-9pdgh\" (UID: \"a7419876-6314-486e-b064-ee186d6cac15\") " pod="openshift-insights/insights-operator-5885fdb759-9pdgh"
Jul 30 17:31:12.617752 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.617727    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7419876-6314-486e-b064-ee186d6cac15-serving-cert\") pod \"insights-operator-5885fdb759-9pdgh\" (UID: \"a7419876-6314-486e-b064-ee186d6cac15\") " pod="openshift-insights/insights-operator-5885fdb759-9pdgh"
Jul 30 17:31:12.618183 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.618156    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7419876-6314-486e-b064-ee186d6cac15-tmp\") pod \"insights-operator-5885fdb759-9pdgh\" (UID: \"a7419876-6314-486e-b064-ee186d6cac15\") " pod="openshift-insights/insights-operator-5885fdb759-9pdgh"
Jul 30 17:31:12.618255 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.618212    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7419876-6314-486e-b064-ee186d6cac15-service-ca-bundle\") pod \"insights-operator-5885fdb759-9pdgh\" (UID: \"a7419876-6314-486e-b064-ee186d6cac15\") " pod="openshift-insights/insights-operator-5885fdb759-9pdgh"
Jul 30 17:31:12.618293 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.618252    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a7419876-6314-486e-b064-ee186d6cac15-snapshots\") pod \"insights-operator-5885fdb759-9pdgh\" (UID: \"a7419876-6314-486e-b064-ee186d6cac15\") " pod="openshift-insights/insights-operator-5885fdb759-9pdgh"
Jul 30 17:31:12.618486 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.618470    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7419876-6314-486e-b064-ee186d6cac15-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-9pdgh\" (UID: \"a7419876-6314-486e-b064-ee186d6cac15\") " pod="openshift-insights/insights-operator-5885fdb759-9pdgh"
Jul 30 17:31:12.620245 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.620224    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7419876-6314-486e-b064-ee186d6cac15-serving-cert\") pod \"insights-operator-5885fdb759-9pdgh\" (UID: \"a7419876-6314-486e-b064-ee186d6cac15\") " pod="openshift-insights/insights-operator-5885fdb759-9pdgh"
Jul 30 17:31:12.626021 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.626002    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8wh4\" (UniqueName: \"kubernetes.io/projected/a7419876-6314-486e-b064-ee186d6cac15-kube-api-access-p8wh4\") pod \"insights-operator-5885fdb759-9pdgh\" (UID: \"a7419876-6314-486e-b064-ee186d6cac15\") " pod="openshift-insights/insights-operator-5885fdb759-9pdgh"
Jul 30 17:31:12.680470 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.680378    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:12.725799 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.725755    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-9pdgh"
Jul 30 17:31:12.847342 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:12.847313    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-9pdgh"]
Jul 30 17:31:12.852963 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:31:12.852848    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7419876_6314_486e_b064_ee186d6cac15.slice/crio-ce807ba8d848ef61a553b030d868b92d706290c213fbbf5666d122636666ec91 WatchSource:0}: Error finding container ce807ba8d848ef61a553b030d868b92d706290c213fbbf5666d122636666ec91: Status 404 returned error can't find the container with id ce807ba8d848ef61a553b030d868b92d706290c213fbbf5666d122636666ec91
Jul 30 17:31:13.031297 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:13.031257    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-9pdgh" event={"ID":"a7419876-6314-486e-b064-ee186d6cac15","Type":"ContainerStarted","Data":"ce807ba8d848ef61a553b030d868b92d706290c213fbbf5666d122636666ec91"}
Jul 30 17:31:14.733460 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:14.733422    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:14.814918 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:14.814864    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:16.041010 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:16.040975    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-9pdgh" event={"ID":"a7419876-6314-486e-b064-ee186d6cac15","Type":"ContainerStarted","Data":"e94d79a1379faccd24e032855b61f20d2c99e88055828b33abfae3fe5b83b988"}
Jul 30 17:31:16.060988 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:16.060926    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-9pdgh" podStartSLOduration=1.910683728 podStartE2EDuration="4.060907342s" totalImagesPullingTime="2.150223614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:31:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:31:16.059761021 +0000 UTC m=+250.513325011" watchObservedRunningTime="2026-07-30 17:31:16.060907342 +0000 UTC m=+250.514471334"
Jul 30 17:31:16.491520 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:16.491453    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-czd4t"]
Jul 30 17:31:16.493850 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:16.493832    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:16.496203 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:16.496181    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-85bdp\""
Jul 30 17:31:16.496426 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:16.496410    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 30 17:31:16.496509 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:16.496443    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 30 17:31:16.510574 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:16.510549    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-czd4t"]
Jul 30 17:31:16.526082 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:16.526053    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:16.547519 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:16.547493    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c01212b7-d88c-4819-a9f2-48e7c8222e23-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:16.547804 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:16.547775    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c01212b7-d88c-4819-a9f2-48e7c8222e23-crio-socket\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:16.547932 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:16.547910    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c01212b7-d88c-4819-a9f2-48e7c8222e23-data-volume\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:16.547993 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:16.547934    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c01212b7-d88c-4819-a9f2-48e7c8222e23-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:16.547993 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:16.547969    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fstk8\" (UniqueName: \"kubernetes.io/projected/c01212b7-d88c-4819-a9f2-48e7c8222e23-kube-api-access-fstk8\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:16.648851 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:16.648803    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c01212b7-d88c-4819-a9f2-48e7c8222e23-data-volume\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:16.648851 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:16.648853    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c01212b7-d88c-4819-a9f2-48e7c8222e23-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:16.649053 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:16.648874    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fstk8\" (UniqueName: \"kubernetes.io/projected/c01212b7-d88c-4819-a9f2-48e7c8222e23-kube-api-access-fstk8\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:16.649053 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:16.648972    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 17:31:16.649053 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:16.648990    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c01212b7-d88c-4819-a9f2-48e7c8222e23-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:16.649053 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:16.649050    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c01212b7-d88c-4819-a9f2-48e7c8222e23-insights-runtime-extractor-tls podName:c01212b7-d88c-4819-a9f2-48e7c8222e23 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:17.149029003 +0000 UTC m=+251.602592996 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c01212b7-d88c-4819-a9f2-48e7c8222e23-insights-runtime-extractor-tls") pod "insights-runtime-extractor-czd4t" (UID: "c01212b7-d88c-4819-a9f2-48e7c8222e23") : secret "insights-runtime-extractor-tls" not found
Jul 30 17:31:16.649220 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:16.649063    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 17:31:16.649220 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:16.649095    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c01212b7-d88c-4819-a9f2-48e7c8222e23-crio-socket\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:16.649220 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:16.649106    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c01212b7-d88c-4819-a9f2-48e7c8222e23-kube-rbac-proxy-cm podName:c01212b7-d88c-4819-a9f2-48e7c8222e23 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:17.149095009 +0000 UTC m=+251.602658977 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c01212b7-d88c-4819-a9f2-48e7c8222e23-kube-rbac-proxy-cm") pod "insights-runtime-extractor-czd4t" (UID: "c01212b7-d88c-4819-a9f2-48e7c8222e23") : configmap "kube-rbac-proxy" not found
Jul 30 17:31:16.649220 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:16.649148    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c01212b7-d88c-4819-a9f2-48e7c8222e23-crio-socket\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:16.649220 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:16.649188    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c01212b7-d88c-4819-a9f2-48e7c8222e23-data-volume\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:16.657170 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:16.657148    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fstk8\" (UniqueName: \"kubernetes.io/projected/c01212b7-d88c-4819-a9f2-48e7c8222e23-kube-api-access-fstk8\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:17.152377 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:17.152333    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c01212b7-d88c-4819-a9f2-48e7c8222e23-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:17.152864 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:17.152437    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c01212b7-d88c-4819-a9f2-48e7c8222e23-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:17.152864 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:17.152507    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 17:31:17.152864 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:17.152572    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c01212b7-d88c-4819-a9f2-48e7c8222e23-kube-rbac-proxy-cm podName:c01212b7-d88c-4819-a9f2-48e7c8222e23 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:18.152555898 +0000 UTC m=+252.606119871 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c01212b7-d88c-4819-a9f2-48e7c8222e23-kube-rbac-proxy-cm") pod "insights-runtime-extractor-czd4t" (UID: "c01212b7-d88c-4819-a9f2-48e7c8222e23") : configmap "kube-rbac-proxy" not found
Jul 30 17:31:17.152864 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:17.152629    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 17:31:17.152864 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:17.152711    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c01212b7-d88c-4819-a9f2-48e7c8222e23-insights-runtime-extractor-tls podName:c01212b7-d88c-4819-a9f2-48e7c8222e23 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:18.152692775 +0000 UTC m=+252.606256750 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c01212b7-d88c-4819-a9f2-48e7c8222e23-insights-runtime-extractor-tls") pod "insights-runtime-extractor-czd4t" (UID: "c01212b7-d88c-4819-a9f2-48e7c8222e23") : secret "insights-runtime-extractor-tls" not found
Jul 30 17:31:17.450791 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:17.450715    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:17.548619 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:17.548592    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:17.649376 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:17.649351    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:17.749876 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:17.749850    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:17.848069 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:17.848045    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:17.857173 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:17.857151    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs\") pod \"network-metrics-daemon-lwrkg\" (UID: \"845cc472-f650-4adc-81b0-94b11e29ecfd\") " pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:31:17.857306 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:17.857291    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 30 17:31:17.857355 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:17.857345    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs podName:845cc472-f650-4adc-81b0-94b11e29ecfd nodeName:}" failed. No retries permitted until 2026-07-30 17:33:19.857331117 +0000 UTC m=+374.310895086 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs") pod "network-metrics-daemon-lwrkg" (UID: "845cc472-f650-4adc-81b0-94b11e29ecfd") : secret "metrics-daemon-secret" not found
Jul 30 17:31:18.159160 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:18.159074    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c01212b7-d88c-4819-a9f2-48e7c8222e23-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:18.159160 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:18.159124    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c01212b7-d88c-4819-a9f2-48e7c8222e23-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:18.159544 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:18.159220    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 17:31:18.159544 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:18.159247    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 17:31:18.159544 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:18.159298    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c01212b7-d88c-4819-a9f2-48e7c8222e23-kube-rbac-proxy-cm podName:c01212b7-d88c-4819-a9f2-48e7c8222e23 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:20.159278886 +0000 UTC m=+254.612842858 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c01212b7-d88c-4819-a9f2-48e7c8222e23-kube-rbac-proxy-cm") pod "insights-runtime-extractor-czd4t" (UID: "c01212b7-d88c-4819-a9f2-48e7c8222e23") : configmap "kube-rbac-proxy" not found
Jul 30 17:31:18.159544 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:18.159313    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c01212b7-d88c-4819-a9f2-48e7c8222e23-insights-runtime-extractor-tls podName:c01212b7-d88c-4819-a9f2-48e7c8222e23 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:20.159306778 +0000 UTC m=+254.612870747 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c01212b7-d88c-4819-a9f2-48e7c8222e23-insights-runtime-extractor-tls") pod "insights-runtime-extractor-czd4t" (UID: "c01212b7-d88c-4819-a9f2-48e7c8222e23") : secret "insights-runtime-extractor-tls" not found
Jul 30 17:31:18.204410 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:18.204380    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:18.305645 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:18.305614    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:18.443240 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:18.443162    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:18.541423 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:18.541395    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:18.664521 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:18.664498    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mxb44_d9f46378-294e-4eda-a3c4-08073cac32f2/dns-node-resolver/0.log"
Jul 30 17:31:19.461296 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:19.461263    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hkwdn_8267208d-1805-4410-bf0c-595b1f1604f1/node-ca/0.log"
Jul 30 17:31:19.771346 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:19.771322    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:19.833474 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:19.833451    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:20.094386 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:20.094302    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:20.172340 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:20.172313    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:20.173300 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:20.173281    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c01212b7-d88c-4819-a9f2-48e7c8222e23-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:20.173381 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:20.173366    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c01212b7-d88c-4819-a9f2-48e7c8222e23-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:20.173436 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:20.173408    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 17:31:20.173478 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:20.173467    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 17:31:20.173515 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:20.173490    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c01212b7-d88c-4819-a9f2-48e7c8222e23-kube-rbac-proxy-cm podName:c01212b7-d88c-4819-a9f2-48e7c8222e23 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:24.173469702 +0000 UTC m=+258.627033671 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c01212b7-d88c-4819-a9f2-48e7c8222e23-kube-rbac-proxy-cm") pod "insights-runtime-extractor-czd4t" (UID: "c01212b7-d88c-4819-a9f2-48e7c8222e23") : configmap "kube-rbac-proxy" not found
Jul 30 17:31:20.173515 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:20.173511    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c01212b7-d88c-4819-a9f2-48e7c8222e23-insights-runtime-extractor-tls podName:c01212b7-d88c-4819-a9f2-48e7c8222e23 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:24.173501181 +0000 UTC m=+258.627065149 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c01212b7-d88c-4819-a9f2-48e7c8222e23-insights-runtime-extractor-tls") pod "insights-runtime-extractor-czd4t" (UID: "c01212b7-d88c-4819-a9f2-48e7c8222e23") : secret "insights-runtime-extractor-tls" not found
Jul 30 17:31:20.263061 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:20.263032    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-brvhl_9feb0d2f-7be2-4f8a-b867-289f917ca8a4/serve-healthcheck-canary/0.log"
Jul 30 17:31:21.060780 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:21.060751    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:23.625288 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:23.625256    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:23.725725 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:23.725697    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:23.826169 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:23.826136    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:24.062787 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:24.062756    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:24.163243 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:24.163217    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:24.204481 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:24.204435    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c01212b7-d88c-4819-a9f2-48e7c8222e23-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:24.204481 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:24.204483    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c01212b7-d88c-4819-a9f2-48e7c8222e23-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:24.204724 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:24.204576    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 17:31:24.204724 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:24.204617    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 17:31:24.204724 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:24.204636    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c01212b7-d88c-4819-a9f2-48e7c8222e23-kube-rbac-proxy-cm podName:c01212b7-d88c-4819-a9f2-48e7c8222e23 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:32.20461963 +0000 UTC m=+266.658183601 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c01212b7-d88c-4819-a9f2-48e7c8222e23-kube-rbac-proxy-cm") pod "insights-runtime-extractor-czd4t" (UID: "c01212b7-d88c-4819-a9f2-48e7c8222e23") : configmap "kube-rbac-proxy" not found
Jul 30 17:31:24.204724 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:24.204677    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c01212b7-d88c-4819-a9f2-48e7c8222e23-insights-runtime-extractor-tls podName:c01212b7-d88c-4819-a9f2-48e7c8222e23 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:32.204660251 +0000 UTC m=+266.658224220 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c01212b7-d88c-4819-a9f2-48e7c8222e23-insights-runtime-extractor-tls") pod "insights-runtime-extractor-czd4t" (UID: "c01212b7-d88c-4819-a9f2-48e7c8222e23") : secret "insights-runtime-extractor-tls" not found
Jul 30 17:31:24.270833 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:24.270791    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:24.374172 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:24.374107    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:25.311947 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:25.311917    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:25.712188 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:25.712106    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:25.910057 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:25.910024    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:26.136714 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:26.136683    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:26.503495 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:26.503461    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:27.055394 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:27.055364    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:27.156456 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:27.156422    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:27.498532 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:27.498491    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:27.551372 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:27.551348    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:27.699663 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:27.699633    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:28.301176 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:28.301141    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:28.909178 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:28.909149    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:29.103225 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:29.103190    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:29.303936 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:29.303901    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:29.501542 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:29.501510    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:29.702464 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:29.702386    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:29.758351 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:29.758325    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:30.500404 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:30.500372    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:30.903778 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:30.903700    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:31.096920 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:31.096887    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:31.306061 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:31.306034    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:31.506364 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:31.506332    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:32.269174 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:32.269143    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c01212b7-d88c-4819-a9f2-48e7c8222e23-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:32.269174 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:32.269183    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c01212b7-d88c-4819-a9f2-48e7c8222e23-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:32.269379 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:32.269284    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 17:31:32.269379 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:32.269349    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c01212b7-d88c-4819-a9f2-48e7c8222e23-insights-runtime-extractor-tls podName:c01212b7-d88c-4819-a9f2-48e7c8222e23 nodeName:}" failed. No retries permitted until 2026-07-30 17:31:48.269333317 +0000 UTC m=+282.722897287 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c01212b7-d88c-4819-a9f2-48e7c8222e23-insights-runtime-extractor-tls") pod "insights-runtime-extractor-czd4t" (UID: "c01212b7-d88c-4819-a9f2-48e7c8222e23") : secret "insights-runtime-extractor-tls" not found
Jul 30 17:31:32.269641 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:32.269622    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c01212b7-d88c-4819-a9f2-48e7c8222e23-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:32.498447 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:32.498412    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:32.697418 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:32.697340    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:33.118261 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:33.118228    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:33.299215 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:33.299184    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:33.699510 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:33.699479    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:34.109627 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:34.109591    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:34.327295 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:34.327260    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:34.767979 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:34.767934    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp" podUID="6f927e85-aa58-4a84-a5d2-038558a187f4" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 30 17:31:38.788215 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:38.788182    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b7488fbb5-86nng"]
Jul 30 17:31:38.790322 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:38.790305    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:38.792742 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:38.792713    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 30 17:31:38.792873 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:38.792788    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bns8l\""
Jul 30 17:31:38.792873 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:38.792788    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 30 17:31:38.793322 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:38.793308    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 30 17:31:38.799200 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:38.799182    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 30 17:31:38.806955 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:38.806926    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b7488fbb5-86nng"]
Jul 30 17:31:38.848486 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:38.848448    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:38.918340 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:38.918300    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-trusted-ca\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:38.918340 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:38.918332    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh2s4\" (UniqueName: \"kubernetes.io/projected/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-kube-api-access-gh2s4\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:38.918550 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:38.918390    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-ca-trust-extracted\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:38.918550 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:38.918428    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-registry-certificates\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:38.918550 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:38.918504    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-installation-pull-secrets\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:38.918550 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:38.918525    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-image-registry-private-configuration\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:38.918550 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:38.918542    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-bound-sa-token\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:38.918722 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:38.918557    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-registry-tls\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:38.947199 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:38.947170    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:39.019455 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:39.019418    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-trusted-ca\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:39.019641 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:39.019456    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh2s4\" (UniqueName: \"kubernetes.io/projected/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-kube-api-access-gh2s4\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:39.019641 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:39.019531    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-ca-trust-extracted\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:39.019641 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:39.019584    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-registry-certificates\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:39.019641 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:39.019613    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-installation-pull-secrets\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:39.019868 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:39.019642    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-image-registry-private-configuration\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:39.019868 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:39.019671    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-bound-sa-token\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:39.019868 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:39.019697    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-registry-tls\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:39.025130 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:39.020392    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-ca-trust-extracted\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:39.025130 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:39.021366    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-registry-certificates\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:39.025130 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:39.021376    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-trusted-ca\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:39.025130 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:39.023126    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-image-registry-private-configuration\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:39.025130 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:39.023281    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-registry-tls\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:39.026777 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:39.026747    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-installation-pull-secrets\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:39.028916 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:39.028889    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-bound-sa-token\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:39.029057 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:39.029039    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh2s4\" (UniqueName: \"kubernetes.io/projected/72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8-kube-api-access-gh2s4\") pod \"image-registry-5b7488fbb5-86nng\" (UID: \"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8\") " pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:39.099436 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:39.099340    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:39.229428 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:39.229395    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b7488fbb5-86nng"]
Jul 30 17:31:39.232398 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:31:39.232364    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72f7ce7b_11e8_47a9_a9ae_a0cd782c46c8.slice/crio-3fb8729a7f451a194be18695e542b5269549bfe447637709fc76f389be88ed45 WatchSource:0}: Error finding container 3fb8729a7f451a194be18695e542b5269549bfe447637709fc76f389be88ed45: Status 404 returned error can't find the container with id 3fb8729a7f451a194be18695e542b5269549bfe447637709fc76f389be88ed45
Jul 30 17:31:40.109771 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:40.109736    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b7488fbb5-86nng" event={"ID":"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8","Type":"ContainerStarted","Data":"f7861b8161a8b784caec85479cba6a018eedc6500acebb1d1818570b9bd2998a"}
Jul 30 17:31:40.110262 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:40.109780    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b7488fbb5-86nng" event={"ID":"72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8","Type":"ContainerStarted","Data":"3fb8729a7f451a194be18695e542b5269549bfe447637709fc76f389be88ed45"}
Jul 30 17:31:40.110262 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:40.109854    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:31:40.131479 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:40.131432    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5b7488fbb5-86nng" podStartSLOduration=2.131417067 podStartE2EDuration="2.131417067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:31:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:31:40.130478774 +0000 UTC m=+274.584042766" watchObservedRunningTime="2026-07-30 17:31:40.131417067 +0000 UTC m=+274.584981058"
Jul 30 17:31:42.765997 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:42.765967    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:42.822553 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:42.822527    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:44.767080 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:44.767044    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp" podUID="6f927e85-aa58-4a84-a5d2-038558a187f4" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 30 17:31:47.407531 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:47.407450    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:47.789059 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:31:47.789021    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5stgf" podUID="9efc23b9-3f92-42ef-b797-7a8608be85cd"
Jul 30 17:31:48.131308 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:48.131230    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5stgf"
Jul 30 17:31:48.291530 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:48.291483    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c01212b7-d88c-4819-a9f2-48e7c8222e23-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:48.293938 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:48.293917    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c01212b7-d88c-4819-a9f2-48e7c8222e23-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-czd4t\" (UID: \"c01212b7-d88c-4819-a9f2-48e7c8222e23\") " pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:48.302927 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:48.302901    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-czd4t"
Jul 30 17:31:48.434487 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:48.434451    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-czd4t"]
Jul 30 17:31:48.437665 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:31:48.437639    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc01212b7_d88c_4819_a9f2_48e7c8222e23.slice/crio-14a6f1fd9e864de4722dcd0d54b0e9f5878aed5e312b1f3cf6d0b3379b08b302 WatchSource:0}: Error finding container 14a6f1fd9e864de4722dcd0d54b0e9f5878aed5e312b1f3cf6d0b3379b08b302: Status 404 returned error can't find the container with id 14a6f1fd9e864de4722dcd0d54b0e9f5878aed5e312b1f3cf6d0b3379b08b302
Jul 30 17:31:48.616446 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:48.616404    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:49.137176 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:49.137139    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-czd4t" event={"ID":"c01212b7-d88c-4819-a9f2-48e7c8222e23","Type":"ContainerStarted","Data":"14a6f1fd9e864de4722dcd0d54b0e9f5878aed5e312b1f3cf6d0b3379b08b302"}
Jul 30 17:31:51.317901 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:51.317857    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls\") pod \"dns-default-5stgf\" (UID: \"9efc23b9-3f92-42ef-b797-7a8608be85cd\") " pod="openshift-dns/dns-default-5stgf"
Jul 30 17:31:51.320505 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:51.320479    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9efc23b9-3f92-42ef-b797-7a8608be85cd-metrics-tls\") pod \"dns-default-5stgf\" (UID: \"9efc23b9-3f92-42ef-b797-7a8608be85cd\") " pod="openshift-dns/dns-default-5stgf"
Jul 30 17:31:51.434224 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:51.434186    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tjl7j\""
Jul 30 17:31:51.442880 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:51.442851    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5stgf"
Jul 30 17:31:51.591566 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:51.591484    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:51.674408 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:51.674378    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:51.884146 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:51.884062    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:52.747844 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:52.747794    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:52.803399 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:52.803363    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:53.398650 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:53.398621    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5stgf"]
Jul 30 17:31:53.402522 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:31:53.402485    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9efc23b9_3f92_42ef_b797_7a8608be85cd.slice/crio-b67aa8f472ead58fa92d2921c1e97cafbe9fa86c302b07081818cfd9de55e9d0 WatchSource:0}: Error finding container b67aa8f472ead58fa92d2921c1e97cafbe9fa86c302b07081818cfd9de55e9d0: Status 404 returned error can't find the container with id b67aa8f472ead58fa92d2921c1e97cafbe9fa86c302b07081818cfd9de55e9d0
Jul 30 17:31:53.506326 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:53.506287    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:53.569410 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:53.569343    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:53.930192 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:53.930073    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:54.009677 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:54.009606    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:54.154175 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:54.154136    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5stgf" event={"ID":"9efc23b9-3f92-42ef-b797-7a8608be85cd","Type":"ContainerStarted","Data":"b67aa8f472ead58fa92d2921c1e97cafbe9fa86c302b07081818cfd9de55e9d0"}
Jul 30 17:31:54.156303 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:54.156230    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-czd4t" event={"ID":"c01212b7-d88c-4819-a9f2-48e7c8222e23","Type":"ContainerStarted","Data":"1d7b604f48f6a18241a96c8cd9c8e273c56fd0d5b476a688bbed110e4b48cb5b"}
Jul 30 17:31:54.767297 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:54.767260    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp" podUID="6f927e85-aa58-4a84-a5d2-038558a187f4" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 30 17:31:54.767465 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:54.767333    2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp"
Jul 30 17:31:54.767878 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:54.767838    2577 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"3871c7973ef6fe60ce06e67c72d4fbfeb9114c084f2dd7ff477b2278360a0226"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 30 17:31:54.768014 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:54.767900    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp" podUID="6f927e85-aa58-4a84-a5d2-038558a187f4" containerName="service-proxy" containerID="cri-o://3871c7973ef6fe60ce06e67c72d4fbfeb9114c084f2dd7ff477b2278360a0226" gracePeriod=30
Jul 30 17:31:55.162562 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:55.162466    2577 generic.go:356] "Generic (PLEG): container finished" podID="6f927e85-aa58-4a84-a5d2-038558a187f4" containerID="3871c7973ef6fe60ce06e67c72d4fbfeb9114c084f2dd7ff477b2278360a0226" exitCode=2
Jul 30 17:31:55.162562 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:55.162540    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp" event={"ID":"6f927e85-aa58-4a84-a5d2-038558a187f4","Type":"ContainerDied","Data":"3871c7973ef6fe60ce06e67c72d4fbfeb9114c084f2dd7ff477b2278360a0226"}
Jul 30 17:31:57.481550 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:57.481521    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:58.174008 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:58.173976    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5stgf" event={"ID":"9efc23b9-3f92-42ef-b797-7a8608be85cd","Type":"ContainerStarted","Data":"d73017fb545e4d3ddc56a0e61febe40a25998c05f8a01f4cdfe7e7fe8a92c4b8"}
Jul 30 17:31:58.174220 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:58.174034    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5stgf" event={"ID":"9efc23b9-3f92-42ef-b797-7a8608be85cd","Type":"ContainerStarted","Data":"a5afa0fad4b1ed59de03cd2f9a3a357bc037d5a38edc0eb4fa4b1693367d7533"}
Jul 30 17:31:58.174220 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:58.174127    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5stgf"
Jul 30 17:31:58.176801 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:58.176762    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d74fb7447-vm2cp" event={"ID":"6f927e85-aa58-4a84-a5d2-038558a187f4","Type":"ContainerStarted","Data":"e83581f21a02603a27a98fe971d6c677c1bd320cc82a74c31152696a6361680c"}
Jul 30 17:31:58.178790 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:58.178764    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-czd4t" event={"ID":"c01212b7-d88c-4819-a9f2-48e7c8222e23","Type":"ContainerStarted","Data":"d03408effcade1fa7478610915d7ebf1985672eed180fc62d56d7170625d39af"}
Jul 30 17:31:58.193853 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:58.193785    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5stgf" podStartSLOduration=253.400739747 podStartE2EDuration="4m17.193770453s" totalImagesPullingTime="3.793030706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:31:58.192798086 +0000 UTC m=+292.646362078" watchObservedRunningTime="2026-07-30 17:31:58.193770453 +0000 UTC m=+292.647334422"
Jul 30 17:31:59.818259 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:59.818229    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:31:59.992561 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:31:59.992446    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:00.047905 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.047867    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xj656"]
Jul 30 17:32:00.060784 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.060750    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.064748 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.064708    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 30 17:32:00.065305 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.065237    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 30 17:32:00.065305 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.065296    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 30 17:32:00.065483 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.065244    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 30 17:32:00.065483 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.065469    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wfrg5\""
Jul 30 17:32:00.065797 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.065680    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 30 17:32:00.065797 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.065691    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 30 17:32:00.105086 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.105014    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:00.195127 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.195083    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-metrics-client-ca\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.195321 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.195139    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4b5w\" (UniqueName: \"kubernetes.io/projected/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-kube-api-access-v4b5w\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.195321 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.195173    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-node-exporter-accelerators-collector-config\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.195321 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.195204    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.195321 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.195237    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-root\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.195321 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.195290    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-node-exporter-wtmp\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.195321 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.195307    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-node-exporter-tls\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.195321 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.195323    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-node-exporter-textfile\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.195732 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.195408    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-sys\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.203805 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.203761    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:00.296021 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.295977    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-node-exporter-accelerators-collector-config\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.296212 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.296030    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.296212 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.296064    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-root\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.296212 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.296102    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-node-exporter-wtmp\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.296212 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.296130    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-node-exporter-tls\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.296212 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.296155    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-node-exporter-textfile\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.296212 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.296175    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-root\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.296212 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.296204    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-sys\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.296557 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.296230    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-metrics-client-ca\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.296557 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:32:00.296239    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 30 17:32:00.296557 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.296271    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-node-exporter-wtmp\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.296557 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.296278    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4b5w\" (UniqueName: \"kubernetes.io/projected/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-kube-api-access-v4b5w\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.296557 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:32:00.296320    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-node-exporter-tls podName:fc79f2a5-6bc1-44a7-b5ef-5162a3a01940 nodeName:}" failed. No retries permitted until 2026-07-30 17:32:00.796298414 +0000 UTC m=+295.249862403 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-node-exporter-tls") pod "node-exporter-xj656" (UID: "fc79f2a5-6bc1-44a7-b5ef-5162a3a01940") : secret "node-exporter-tls" not found
Jul 30 17:32:00.296557 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.296337    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-sys\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.296891 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.296645    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-node-exporter-accelerators-collector-config\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.296891 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.296645    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-node-exporter-textfile\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.296891 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.296846    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-metrics-client-ca\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.298722 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.298698    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.303477 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.303445    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:00.306140 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.306118    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4b5w\" (UniqueName: \"kubernetes.io/projected/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-kube-api-access-v4b5w\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.483416 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.483336    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:00.583625 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.583586    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:00.801102 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.801067    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-node-exporter-tls\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.803602 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.803579    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fc79f2a5-6bc1-44a7-b5ef-5162a3a01940-node-exporter-tls\") pod \"node-exporter-xj656\" (UID: \"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940\") " pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.961321 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.961286    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:00.972299 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:00.972271    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xj656"
Jul 30 17:32:00.982525 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:32:00.982494    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc79f2a5_6bc1_44a7_b5ef_5162a3a01940.slice/crio-d3fa3a1abe7337e7f8dbbf78bcd5d76123b66e44ec91981994336ecbd8c9e111 WatchSource:0}: Error finding container d3fa3a1abe7337e7f8dbbf78bcd5d76123b66e44ec91981994336ecbd8c9e111: Status 404 returned error can't find the container with id d3fa3a1abe7337e7f8dbbf78bcd5d76123b66e44ec91981994336ecbd8c9e111
Jul 30 17:32:01.117398 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.117313    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5b7488fbb5-86nng"
Jul 30 17:32:01.118964 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.118909    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:01.190518 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.190481    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xj656" event={"ID":"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940","Type":"ContainerStarted","Data":"d3fa3a1abe7337e7f8dbbf78bcd5d76123b66e44ec91981994336ecbd8c9e111"}
Jul 30 17:32:01.192673 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.192639    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-czd4t" event={"ID":"c01212b7-d88c-4819-a9f2-48e7c8222e23","Type":"ContainerStarted","Data":"0b491d60f0caeedd693502078355e3356dbc42fc435186231c29a606017d7503"}
Jul 30 17:32:01.272405 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.272180    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 17:32:01.275670 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.275645    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.283650 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:32:01.283619    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-kube-rbac-proxy-metric\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" type="*v1.Secret"
Jul 30 17:32:01.283829 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:32:01.283631    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-main-dockercfg-rtw87\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rtw87\"" type="*v1.Secret"
Jul 30 17:32:01.283829 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:32:01.283622    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-main-cluster-tls-config\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" type="*v1.Secret"
Jul 30 17:32:01.283829 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:32:01.283669    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-main-tls-assets-0\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" type="*v1.Secret"
Jul 30 17:32:01.283829 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.283680    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 30 17:32:01.283829 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:32:01.283722    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-main-generated\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" type="*v1.Secret"
Jul 30 17:32:01.283829 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:32:01.283676    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-kube-rbac-proxy\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" type="*v1.Secret"
Jul 30 17:32:01.283829 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.283681    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 30 17:32:01.283829 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:32:01.283773    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-main-web-config\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" type="*v1.Secret"
Jul 30 17:32:01.283829 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.283797    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 30 17:32:01.306187 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.306153    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:01.309602 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.309561    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-czd4t" podStartSLOduration=33.083664694 podStartE2EDuration="45.309546352s" totalImagesPullingTime="12.225881658s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 17:31:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:32:01.276456775 +0000 UTC m=+295.730020766" watchObservedRunningTime="2026-07-30 17:32:01.309546352 +0000 UTC m=+295.763110342"
Jul 30 17:32:01.310511 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.310492    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 17:32:01.406540 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.406440    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd67cfff-a350-42fd-ac86-da5d41f820b6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.406540 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.406487    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-web-config\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.406540 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.406514    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.406540 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.406530    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.406789 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.406578    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd67cfff-a350-42fd-ac86-da5d41f820b6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.406789 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.406617    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.406789 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.406659    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dd67cfff-a350-42fd-ac86-da5d41f820b6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.406789 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.406693    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.406789 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.406721    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-config-volume\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.406789 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.406760    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dd67cfff-a350-42fd-ac86-da5d41f820b6-config-out\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.406789 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.406775    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlwqs\" (UniqueName: \"kubernetes.io/projected/dd67cfff-a350-42fd-ac86-da5d41f820b6-kube-api-access-xlwqs\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.407042 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.406805    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dd67cfff-a350-42fd-ac86-da5d41f820b6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.407042 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.406857    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.475129 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.475094    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:01.508212 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.508175    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.508371 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.508217    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-config-volume\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.508371 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.508253    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dd67cfff-a350-42fd-ac86-da5d41f820b6-config-out\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.508371 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.508270    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlwqs\" (UniqueName: \"kubernetes.io/projected/dd67cfff-a350-42fd-ac86-da5d41f820b6-kube-api-access-xlwqs\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.508371 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.508306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dd67cfff-a350-42fd-ac86-da5d41f820b6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.508371 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.508336    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.508371 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.508365    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd67cfff-a350-42fd-ac86-da5d41f820b6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.508674 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.508394    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-web-config\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.508674 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.508428    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.508674 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.508449    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.508674 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.508474    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd67cfff-a350-42fd-ac86-da5d41f820b6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.508674 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.508504    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.508674 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.508530    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dd67cfff-a350-42fd-ac86-da5d41f820b6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.509069 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.508893    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dd67cfff-a350-42fd-ac86-da5d41f820b6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.509445 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.509423    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd67cfff-a350-42fd-ac86-da5d41f820b6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.509529 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.509507    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd67cfff-a350-42fd-ac86-da5d41f820b6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.510752 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.510728    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dd67cfff-a350-42fd-ac86-da5d41f820b6-config-out\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.511096 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.511073    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.511177 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.511163    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:01.517990 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:01.517962    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlwqs\" (UniqueName: \"kubernetes.io/projected/dd67cfff-a350-42fd-ac86-da5d41f820b6-kube-api-access-xlwqs\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:02.042660 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:02.042633    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:02.121572 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:02.121543    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rtw87\""
Jul 30 17:32:02.152332 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:02.152303    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:02.197471 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:02.197436    2577 generic.go:356] "Generic (PLEG): container finished" podID="fc79f2a5-6bc1-44a7-b5ef-5162a3a01940" containerID="b3afe37cabf3e8c4a138b8386ac9272f8e587fd251194ec3336ebe699683b14d" exitCode=0
Jul 30 17:32:02.197662 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:02.197537    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xj656" event={"ID":"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940","Type":"ContainerDied","Data":"b3afe37cabf3e8c4a138b8386ac9272f8e587fd251194ec3336ebe699683b14d"}
Jul 30 17:32:02.257124 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:02.257102    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 30 17:32:02.261278 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:02.261254    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:02.352911 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:02.352879    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 30 17:32:02.361435 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:02.361405    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:02.475956 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:02.475925    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 30 17:32:02.480960 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:02.480897    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dd67cfff-a350-42fd-ac86-da5d41f820b6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:02.509368 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:32:02.509334    2577 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-web-config: failed to sync secret cache: timed out waiting for the condition
Jul 30 17:32:02.509531 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:32:02.509424    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-web-config podName:dd67cfff-a350-42fd-ac86-da5d41f820b6 nodeName:}" failed. No retries permitted until 2026-07-30 17:32:03.009404636 +0000 UTC m=+297.462968608 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "web-config" (UniqueName: "kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-web-config") pod "alertmanager-main-0" (UID: "dd67cfff-a350-42fd-ac86-da5d41f820b6") : failed to sync secret cache: timed out waiting for the condition
Jul 30 17:32:02.509531 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:32:02.509343    2577 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-generated: failed to sync secret cache: timed out waiting for the condition
Jul 30 17:32:02.509531 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:32:02.509486    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-config-volume podName:dd67cfff-a350-42fd-ac86-da5d41f820b6 nodeName:}" failed. No retries permitted until 2026-07-30 17:32:03.009473162 +0000 UTC m=+297.463037131 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-volume" (UniqueName: "kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-config-volume") pod "alertmanager-main-0" (UID: "dd67cfff-a350-42fd-ac86-da5d41f820b6") : failed to sync secret cache: timed out waiting for the condition
Jul 30 17:32:02.509531 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:32:02.509343    2577 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-kube-rbac-proxy: failed to sync secret cache: timed out waiting for the condition
Jul 30 17:32:02.509531 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:32:02.509519    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-kube-rbac-proxy podName:dd67cfff-a350-42fd-ac86-da5d41f820b6 nodeName:}" failed. No retries permitted until 2026-07-30 17:32:03.009509961 +0000 UTC m=+297.463073929 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-kube-rbac-proxy" (UniqueName: "kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-kube-rbac-proxy") pod "alertmanager-main-0" (UID: "dd67cfff-a350-42fd-ac86-da5d41f820b6") : failed to sync secret cache: timed out waiting for the condition
Jul 30 17:32:02.572959 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:02.572927    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 30 17:32:02.621027 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:02.620994    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 30 17:32:02.625457 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:02.625439    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 30 17:32:02.952000 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:02.951966    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:03.024144 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.024117    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-config-volume\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:03.024306 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.024180    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-web-config\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:03.024306 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.024211    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:03.027283 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.027253    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:03.027517 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.027360    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-config-volume\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:03.027517 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.027363    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-web-config\") pod \"alertmanager-main-0\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:03.052861 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.052828    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:03.083948 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.083914    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:32:03.105061 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:32:03.105020    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-lwrkg" podUID="845cc472-f650-4adc-81b0-94b11e29ecfd"
Jul 30 17:32:03.152862 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.152762    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:03.194303 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.194271    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"]
Jul 30 17:32:03.198886 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.198863    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.201337 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.201308    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 30 17:32:03.201508 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.201406    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 30 17:32:03.201508 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.201475    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 30 17:32:03.201508 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.201503    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-hfdpn\""
Jul 30 17:32:03.201688 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.201623    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-7aqk806l23k4r\""
Jul 30 17:32:03.201739 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.201693    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 30 17:32:03.201793 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.201762    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 30 17:32:03.203508 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.203450    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xj656" event={"ID":"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940","Type":"ContainerStarted","Data":"fa5cf3f2a94a410247c41504eccca8292006f592f50c50765dcf309d7cd452d0"}
Jul 30 17:32:03.203600 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.203504    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xj656" event={"ID":"fc79f2a5-6bc1-44a7-b5ef-5162a3a01940","Type":"ContainerStarted","Data":"9a7e70fdf373b563d863f6cc00bcc622e59e0e48f8df4b06ed301aeda69a5f68"}
Jul 30 17:32:03.212121 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.212095    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"]
Jul 30 17:32:03.227129 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.227060    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 17:32:03.229604 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:32:03.229579    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd67cfff_a350_42fd_ac86_da5d41f820b6.slice/crio-9b61989e1ad727216e7975eaea074fcb58db81b6feffccd9ddd3e5ed31956076 WatchSource:0}: Error finding container 9b61989e1ad727216e7975eaea074fcb58db81b6feffccd9ddd3e5ed31956076: Status 404 returned error can't find the container with id 9b61989e1ad727216e7975eaea074fcb58db81b6feffccd9ddd3e5ed31956076
Jul 30 17:32:03.245789 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.245743    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xj656" podStartSLOduration=2.17660282 podStartE2EDuration="3.245725894s" totalImagesPullingTime="995.331243ms" totalInitContainerRuntime="73.791831ms" isStatefulPod=true podCreationTimestamp="2026-07-30 17:32:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:32:03.244663635 +0000 UTC m=+297.698227628" watchObservedRunningTime="2026-07-30 17:32:03.245725894 +0000 UTC m=+297.699289884"
Jul 30 17:32:03.326194 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.326150    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/036c7f90-b643-4f96-a932-37753f58d745-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.326391 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.326216    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zfnc\" (UniqueName: \"kubernetes.io/projected/036c7f90-b643-4f96-a932-37753f58d745-kube-api-access-6zfnc\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.326452 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.326405    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/036c7f90-b643-4f96-a932-37753f58d745-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.326541 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.326509    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/036c7f90-b643-4f96-a932-37753f58d745-secret-thanos-querier-tls\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.326594 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.326567    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/036c7f90-b643-4f96-a932-37753f58d745-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.326649 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.326609    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/036c7f90-b643-4f96-a932-37753f58d745-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.326649 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.326639    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/036c7f90-b643-4f96-a932-37753f58d745-metrics-client-ca\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.326747 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.326688    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/036c7f90-b643-4f96-a932-37753f58d745-secret-grpc-tls\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.387907 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.387874    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:03.428063 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.428015    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/036c7f90-b643-4f96-a932-37753f58d745-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.428192 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.428076    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/036c7f90-b643-4f96-a932-37753f58d745-secret-thanos-querier-tls\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.428192 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.428108    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/036c7f90-b643-4f96-a932-37753f58d745-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.428192 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.428134    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/036c7f90-b643-4f96-a932-37753f58d745-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.428192 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.428161    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/036c7f90-b643-4f96-a932-37753f58d745-metrics-client-ca\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.428440 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.428298    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/036c7f90-b643-4f96-a932-37753f58d745-secret-grpc-tls\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.428440 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.428376    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/036c7f90-b643-4f96-a932-37753f58d745-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.428440 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.428409    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zfnc\" (UniqueName: \"kubernetes.io/projected/036c7f90-b643-4f96-a932-37753f58d745-kube-api-access-6zfnc\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.429113 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.429066    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/036c7f90-b643-4f96-a932-37753f58d745-metrics-client-ca\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.431094 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.431069    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/036c7f90-b643-4f96-a932-37753f58d745-secret-thanos-querier-tls\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.431229 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.431203    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/036c7f90-b643-4f96-a932-37753f58d745-secret-grpc-tls\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.431336 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.431206    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/036c7f90-b643-4f96-a932-37753f58d745-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.431336 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.431294    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/036c7f90-b643-4f96-a932-37753f58d745-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.431440 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.431424    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/036c7f90-b643-4f96-a932-37753f58d745-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.431571 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.431553    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/036c7f90-b643-4f96-a932-37753f58d745-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.437017 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.436998    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zfnc\" (UniqueName: \"kubernetes.io/projected/036c7f90-b643-4f96-a932-37753f58d745-kube-api-access-6zfnc\") pod \"thanos-querier-f8c75f8bc-82jb9\" (UID: \"036c7f90-b643-4f96-a932-37753f58d745\") " pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.509082 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.509052    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:03.652676 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.652642    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"]
Jul 30 17:32:03.656254 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:32:03.656230    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod036c7f90_b643_4f96_a932_37753f58d745.slice/crio-13d4ffbabd3ffeb34ce05815b636ea922dcf4230cc31583e850ae40136c4613d WatchSource:0}: Error finding container 13d4ffbabd3ffeb34ce05815b636ea922dcf4230cc31583e850ae40136c4613d: Status 404 returned error can't find the container with id 13d4ffbabd3ffeb34ce05815b636ea922dcf4230cc31583e850ae40136c4613d
Jul 30 17:32:03.660358 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:03.660328    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:04.209689 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:04.209623    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dd67cfff-a350-42fd-ac86-da5d41f820b6","Type":"ContainerStarted","Data":"9b61989e1ad727216e7975eaea074fcb58db81b6feffccd9ddd3e5ed31956076"}
Jul 30 17:32:04.210952 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:04.210916    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9" event={"ID":"036c7f90-b643-4f96-a932-37753f58d745","Type":"ContainerStarted","Data":"13d4ffbabd3ffeb34ce05815b636ea922dcf4230cc31583e850ae40136c4613d"}
Jul 30 17:32:04.781541 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:04.781501    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:04.889787 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:04.889705    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:05.216504 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:05.216416    2577 generic.go:356] "Generic (PLEG): container finished" podID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerID="849a5eea89be50a52d41542a56d11c093b10001a8e0b6732a2c9d9c5bb404357" exitCode=0
Jul 30 17:32:05.217027 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:05.216506    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dd67cfff-a350-42fd-ac86-da5d41f820b6","Type":"ContainerDied","Data":"849a5eea89be50a52d41542a56d11c093b10001a8e0b6732a2c9d9c5bb404357"}
Jul 30 17:32:05.905100 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:05.905009    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:05.938260 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:05.938237    2577 kubelet.go:1648] "Image garbage collection succeeded"
Jul 30 17:32:06.006383 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:06.006353    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:06.222057 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:06.221892    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9" event={"ID":"036c7f90-b643-4f96-a932-37753f58d745","Type":"ContainerStarted","Data":"b5818ac4708811959e0e0fbda6eeec120f978b00e55a5645d77903799e7a730e"}
Jul 30 17:32:06.222057 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:06.221959    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9" event={"ID":"036c7f90-b643-4f96-a932-37753f58d745","Type":"ContainerStarted","Data":"e40814f3ea347c0a6aa38fd43456384e6c174be12898e17502783c339439d5d8"}
Jul 30 17:32:06.222057 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:06.221986    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9" event={"ID":"036c7f90-b643-4f96-a932-37753f58d745","Type":"ContainerStarted","Data":"0170e4dc23d6ecc00dc95f085c1553e2dcc300ff98a4bcafd530f7c5449b9a19"}
Jul 30 17:32:06.345751 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:06.345663    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:08.184402 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:08.184370    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5stgf"
Jul 30 17:32:08.232154 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:08.232115    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9" event={"ID":"036c7f90-b643-4f96-a932-37753f58d745","Type":"ContainerStarted","Data":"dae6da804021e12f83c6ac0f49afd503a5136d2cd4f70e88278db730d9a210ed"}
Jul 30 17:32:08.232354 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:08.232202    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9" event={"ID":"036c7f90-b643-4f96-a932-37753f58d745","Type":"ContainerStarted","Data":"3ef3ca629e2a8588605ddd9a3e3f074499d43f1d4fe3554f447bd4cad3eb32c5"}
Jul 30 17:32:08.232354 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:08.232245    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:08.232354 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:08.232280    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9" event={"ID":"036c7f90-b643-4f96-a932-37753f58d745","Type":"ContainerStarted","Data":"90a551db805318411e97749ca7b347fc4ae5d8867ed1857e1174a4cb16b68a64"}
Jul 30 17:32:08.236046 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:08.236016    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dd67cfff-a350-42fd-ac86-da5d41f820b6","Type":"ContainerStarted","Data":"5bed2935a830b69798c68c01347130950d6d1a942726bfc9538425080c0a3949"}
Jul 30 17:32:08.236174 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:08.236091    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dd67cfff-a350-42fd-ac86-da5d41f820b6","Type":"ContainerStarted","Data":"0277ed63543e61d0d4e173864130beb20d59f7c0f63e266c645c0f4f9bfd5c19"}
Jul 30 17:32:08.236174 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:08.236135    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dd67cfff-a350-42fd-ac86-da5d41f820b6","Type":"ContainerStarted","Data":"4b954ddf41ba2753068f01e379570aa3a53c8783f2cdfe40d039eec8f3dc9541"}
Jul 30 17:32:08.236247 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:08.236175    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dd67cfff-a350-42fd-ac86-da5d41f820b6","Type":"ContainerStarted","Data":"8d8bcfe755faf571cdba2579db49fa9404c65335bf5ea02e9d0e10816c9cfe71"}
Jul 30 17:32:08.236247 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:08.236211    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dd67cfff-a350-42fd-ac86-da5d41f820b6","Type":"ContainerStarted","Data":"652602d76fb179af3fcc9f2f0537f364f19423575369f5a83601a8596ffab4da"}
Jul 30 17:32:08.236308 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:08.236250    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dd67cfff-a350-42fd-ac86-da5d41f820b6","Type":"ContainerStarted","Data":"649514784f1c2cc0f85d4f5ea455ac3e6f7a77f2f1e4df126c7df857c315a4a7"}
Jul 30 17:32:08.256653 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:08.256603    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9" podStartSLOduration=1.854707457 podStartE2EDuration="5.256587735s" totalImagesPullingTime="3.401880278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:32:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:32:08.255778137 +0000 UTC m=+302.709342128" watchObservedRunningTime="2026-07-30 17:32:08.256587735 +0000 UTC m=+302.710151725"
Jul 30 17:32:08.287307 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:08.287235    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.830652424 podStartE2EDuration="7.28721398s" totalImagesPullingTime="3.437200408s" totalInitContainerRuntime="19.361148ms" isStatefulPod=false podCreationTimestamp="2026-07-30 17:32:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:32:08.284713576 +0000 UTC m=+302.738277568" watchObservedRunningTime="2026-07-30 17:32:08.28721398 +0000 UTC m=+302.740777975"
Jul 30 17:32:11.785427 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:11.785393    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:11.882302 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:11.882270    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:12.031124 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:12.031092    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:12.132037 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:12.131956    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:12.450538 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:12.450462    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:12.640146 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:12.640118    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:12.741126 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:12.741055    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:12.802310 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:12.802284    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:12.857484 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:12.857460    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:13.102212 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:13.102176    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:13.166637 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:13.166590    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:13.257355 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:13.257323    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:13.451058 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:13.450989    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:13.507450 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:13.507419    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:14.104943 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:14.104909    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:32:14.244648 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:14.244622    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-f8c75f8bc-82jb9"
Jul 30 17:32:15.117261 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:15.117229    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:15.179393 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:15.179356    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:15.287137 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:15.287099    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:15.388386 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:15.388306    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:17.354490 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:17.354457    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:17.414295 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:17.414254    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:25.877825 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:25.877787    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d97744b78-gnthl"]
Jul 30 17:32:25.892308 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:25.892283    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d97744b78-gnthl"]
Jul 30 17:32:25.892445 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:25.892378    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:25.895104 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:25.895076    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 30 17:32:25.895104 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:25.895079    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 30 17:32:25.895707 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:25.895683    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 30 17:32:25.895707 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:25.895715    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7zw78\""
Jul 30 17:32:25.895984 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:25.895726    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 30 17:32:25.895984 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:25.895849    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 30 17:32:25.896088 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:25.895992    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 30 17:32:25.896153 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:25.896116    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 30 17:32:25.899901 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:25.899882    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 30 17:32:25.929010 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:25.928985    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-service-ca\") pod \"console-6d97744b78-gnthl\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") " pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:25.929168 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:25.929017    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwphz\" (UniqueName: \"kubernetes.io/projected/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-kube-api-access-qwphz\") pod \"console-6d97744b78-gnthl\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") " pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:25.929168 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:25.929049    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-trusted-ca-bundle\") pod \"console-6d97744b78-gnthl\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") " pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:25.929168 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:25.929081    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-console-config\") pod \"console-6d97744b78-gnthl\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") " pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:25.929168 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:25.929117    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-console-serving-cert\") pod \"console-6d97744b78-gnthl\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") " pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:25.929363 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:25.929178    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-console-oauth-config\") pod \"console-6d97744b78-gnthl\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") " pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:25.929363 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:25.929263    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-oauth-serving-cert\") pod \"console-6d97744b78-gnthl\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") " pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:26.030037 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:26.030008    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-oauth-serving-cert\") pod \"console-6d97744b78-gnthl\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") " pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:26.030191 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:26.030061    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-service-ca\") pod \"console-6d97744b78-gnthl\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") " pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:26.030191 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:26.030173    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwphz\" (UniqueName: \"kubernetes.io/projected/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-kube-api-access-qwphz\") pod \"console-6d97744b78-gnthl\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") " pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:26.030278 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:26.030226    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-trusted-ca-bundle\") pod \"console-6d97744b78-gnthl\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") " pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:26.030278 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:26.030262    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-console-config\") pod \"console-6d97744b78-gnthl\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") " pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:26.030353 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:26.030289    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-console-serving-cert\") pod \"console-6d97744b78-gnthl\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") " pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:26.030395 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:26.030350    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-console-oauth-config\") pod \"console-6d97744b78-gnthl\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") " pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:26.030833 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:26.030793    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-service-ca\") pod \"console-6d97744b78-gnthl\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") " pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:26.030961 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:26.030860    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-oauth-serving-cert\") pod \"console-6d97744b78-gnthl\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") " pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:26.031108 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:26.031088    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-console-config\") pod \"console-6d97744b78-gnthl\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") " pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:26.031454 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:26.031436    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-trusted-ca-bundle\") pod \"console-6d97744b78-gnthl\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") " pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:26.033417 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:26.033398    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-console-oauth-config\") pod \"console-6d97744b78-gnthl\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") " pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:26.033552 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:26.033536    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-console-serving-cert\") pod \"console-6d97744b78-gnthl\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") " pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:26.040615 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:26.040597    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwphz\" (UniqueName: \"kubernetes.io/projected/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-kube-api-access-qwphz\") pod \"console-6d97744b78-gnthl\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") " pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:26.203120 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:26.203043    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:26.378975 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:26.378940    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d97744b78-gnthl"]
Jul 30 17:32:26.381460 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:32:26.381431    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode971b04e_381f_4bed_85a4_6ea1fa1c52b5.slice/crio-a7f68151fb32e5751aba3a5c8a41e7791068ddfae7bc8c38e17919cffc909b70 WatchSource:0}: Error finding container a7f68151fb32e5751aba3a5c8a41e7791068ddfae7bc8c38e17919cffc909b70: Status 404 returned error can't find the container with id a7f68151fb32e5751aba3a5c8a41e7791068ddfae7bc8c38e17919cffc909b70
Jul 30 17:32:26.383536 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:26.383520    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 17:32:27.301845 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:27.301780    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d97744b78-gnthl" event={"ID":"e971b04e-381f-4bed-85a4-6ea1fa1c52b5","Type":"ContainerStarted","Data":"a7f68151fb32e5751aba3a5c8a41e7791068ddfae7bc8c38e17919cffc909b70"}
Jul 30 17:32:27.313098 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:27.313048    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:27.372411 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:27.372363    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:27.770369 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:27.770335    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:27.835476 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:27.835439    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:30.313562 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:30.313527    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d97744b78-gnthl" event={"ID":"e971b04e-381f-4bed-85a4-6ea1fa1c52b5","Type":"ContainerStarted","Data":"3cd6e666affae65be4bc2604b687da56ed880256fd888a3d51b89843b427d9cd"}
Jul 30 17:32:30.331355 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:30.331302    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d97744b78-gnthl" podStartSLOduration=2.035258026 podStartE2EDuration="5.331285576s" totalImagesPullingTime="3.29602755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:32:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:32:30.330694818 +0000 UTC m=+324.784258812" watchObservedRunningTime="2026-07-30 17:32:30.331285576 +0000 UTC m=+324.784849598"
Jul 30 17:32:30.896327 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:30.896290    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:30.994072 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:30.994039    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:35.899774 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:35.899689    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:35.961493 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:35.961460    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:36.203932 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:36.203776    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:36.203932 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:36.203881    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:36.208835 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:36.208796    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:36.338069 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:36.338035    2577 generic.go:356] "Generic (PLEG): container finished" podID="a7419876-6314-486e-b064-ee186d6cac15" containerID="e94d79a1379faccd24e032855b61f20d2c99e88055828b33abfae3fe5b83b988" exitCode=0
Jul 30 17:32:36.338229 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:36.338110    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-9pdgh" event={"ID":"a7419876-6314-486e-b064-ee186d6cac15","Type":"ContainerDied","Data":"e94d79a1379faccd24e032855b61f20d2c99e88055828b33abfae3fe5b83b988"}
Jul 30 17:32:36.338558 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:36.338512    2577 scope.go:117] "RemoveContainer" containerID="e94d79a1379faccd24e032855b61f20d2c99e88055828b33abfae3fe5b83b988"
Jul 30 17:32:36.342172 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:36.342152    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:32:37.084227 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:37.084189    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:37.143078 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:37.143047    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:37.343633 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:37.343553    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-9pdgh" event={"ID":"a7419876-6314-486e-b064-ee186d6cac15","Type":"ContainerStarted","Data":"4476e057d0d3fd4d8f7422427055ae700e0287b19943143b13b44f11aec9bf40"}
Jul 30 17:32:37.351462 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:37.351441    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-brvhl_9feb0d2f-7be2-4f8a-b867-289f917ca8a4/serve-healthcheck-canary/0.log"
Jul 30 17:32:38.314332 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:38.314293    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:38.372558 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:38.372522    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:39.291644 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:39.291609    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:39.350623 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:39.350594    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:40.089641 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:40.089607    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:40.149091 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:40.149056    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:40.890314 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:40.890278    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:40.952258 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:40.952229    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:52.117533 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:52.117500    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:52.256100 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:52.256063    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:52.330501 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:52.330465    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:57.771982 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:57.771875    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:32:57.840210 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:32:57.840170    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:33:02.082871 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:02.082838    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:33:02.148020 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:02.147984    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:33:14.257386 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:14.257350    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:33:14.320425 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:14.320392    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:33:19.402460 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:19.402420    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:33:19.601471 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:19.601440    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:33:19.882886 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:19.882848    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs\") pod \"network-metrics-daemon-lwrkg\" (UID: \"845cc472-f650-4adc-81b0-94b11e29ecfd\") " pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:33:19.885256 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:19.885237    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/845cc472-f650-4adc-81b0-94b11e29ecfd-metrics-certs\") pod \"network-metrics-daemon-lwrkg\" (UID: \"845cc472-f650-4adc-81b0-94b11e29ecfd\") " pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:33:20.108171 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:20.108146    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fjwv8\""
Jul 30 17:33:20.116183 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:20.116151    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lwrkg"
Jul 30 17:33:20.249781 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:20.249752    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lwrkg"]
Jul 30 17:33:20.252554 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:33:20.252528    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod845cc472_f650_4adc_81b0_94b11e29ecfd.slice/crio-6cebde58148a916892727908a05c7f810828bfad89b43c49a9ae22a8ff2473b5 WatchSource:0}: Error finding container 6cebde58148a916892727908a05c7f810828bfad89b43c49a9ae22a8ff2473b5: Status 404 returned error can't find the container with id 6cebde58148a916892727908a05c7f810828bfad89b43c49a9ae22a8ff2473b5
Jul 30 17:33:20.421231 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:20.421085    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 17:33:20.502047 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:20.502011    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lwrkg" event={"ID":"845cc472-f650-4adc-81b0-94b11e29ecfd","Type":"ContainerStarted","Data":"6cebde58148a916892727908a05c7f810828bfad89b43c49a9ae22a8ff2473b5"}
Jul 30 17:33:20.502211 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:20.502193    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="alertmanager" containerID="cri-o://649514784f1c2cc0f85d4f5ea455ac3e6f7a77f2f1e4df126c7df857c315a4a7" gracePeriod=120
Jul 30 17:33:20.502258 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:20.502235    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="kube-rbac-proxy-metric" containerID="cri-o://0277ed63543e61d0d4e173864130beb20d59f7c0f63e266c645c0f4f9bfd5c19" gracePeriod=120
Jul 30 17:33:20.502310 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:20.502265    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="config-reloader" containerID="cri-o://652602d76fb179af3fcc9f2f0537f364f19423575369f5a83601a8596ffab4da" gracePeriod=120
Jul 30 17:33:20.502310 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:20.502261    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="kube-rbac-proxy-web" containerID="cri-o://8d8bcfe755faf571cdba2579db49fa9404c65335bf5ea02e9d0e10816c9cfe71" gracePeriod=120
Jul 30 17:33:20.502407 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:20.502321    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="kube-rbac-proxy" containerID="cri-o://4b954ddf41ba2753068f01e379570aa3a53c8783f2cdfe40d039eec8f3dc9541" gracePeriod=120
Jul 30 17:33:20.502407 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:20.502350    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="prom-label-proxy" containerID="cri-o://5bed2935a830b69798c68c01347130950d6d1a942726bfc9538425080c0a3949" gracePeriod=120
Jul 30 17:33:21.507536 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.507504    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lwrkg" event={"ID":"845cc472-f650-4adc-81b0-94b11e29ecfd","Type":"ContainerStarted","Data":"7546bb1aa58098ff648500e8dea3a70ca35a84a7e091ffe251381a6db2903576"}
Jul 30 17:33:21.508014 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.507563    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lwrkg" event={"ID":"845cc472-f650-4adc-81b0-94b11e29ecfd","Type":"ContainerStarted","Data":"eab4cd803f38c2203eb37b34fac35cff7ccc648212bdb4f16b144d27f340bba0"}
Jul 30 17:33:21.510729 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.510702    2577 generic.go:356] "Generic (PLEG): container finished" podID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerID="5bed2935a830b69798c68c01347130950d6d1a942726bfc9538425080c0a3949" exitCode=0
Jul 30 17:33:21.510729 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.510725    2577 generic.go:356] "Generic (PLEG): container finished" podID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerID="4b954ddf41ba2753068f01e379570aa3a53c8783f2cdfe40d039eec8f3dc9541" exitCode=0
Jul 30 17:33:21.510729 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.510731    2577 generic.go:356] "Generic (PLEG): container finished" podID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerID="652602d76fb179af3fcc9f2f0537f364f19423575369f5a83601a8596ffab4da" exitCode=0
Jul 30 17:33:21.510964 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.510737    2577 generic.go:356] "Generic (PLEG): container finished" podID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerID="649514784f1c2cc0f85d4f5ea455ac3e6f7a77f2f1e4df126c7df857c315a4a7" exitCode=0
Jul 30 17:33:21.510964 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.510756    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dd67cfff-a350-42fd-ac86-da5d41f820b6","Type":"ContainerDied","Data":"5bed2935a830b69798c68c01347130950d6d1a942726bfc9538425080c0a3949"}
Jul 30 17:33:21.510964 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.510801    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dd67cfff-a350-42fd-ac86-da5d41f820b6","Type":"ContainerDied","Data":"4b954ddf41ba2753068f01e379570aa3a53c8783f2cdfe40d039eec8f3dc9541"}
Jul 30 17:33:21.510964 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.510851    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dd67cfff-a350-42fd-ac86-da5d41f820b6","Type":"ContainerDied","Data":"652602d76fb179af3fcc9f2f0537f364f19423575369f5a83601a8596ffab4da"}
Jul 30 17:33:21.510964 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.510877    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dd67cfff-a350-42fd-ac86-da5d41f820b6","Type":"ContainerDied","Data":"649514784f1c2cc0f85d4f5ea455ac3e6f7a77f2f1e4df126c7df857c315a4a7"}
Jul 30 17:33:21.528269 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.528227    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lwrkg" podStartSLOduration=374.601031585 podStartE2EDuration="6m15.528212992s" totalImagesPullingTime="927.181407ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:27:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:33:21.527344537 +0000 UTC m=+375.980908528" watchObservedRunningTime="2026-07-30 17:33:21.528212992 +0000 UTC m=+375.981777014"
Jul 30 17:33:21.668803 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.668772    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:33:21.742167 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.742137    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:33:21.743482 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.743463    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:21.801064 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.800982    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dd67cfff-a350-42fd-ac86-da5d41f820b6-tls-assets\") pod \"dd67cfff-a350-42fd-ac86-da5d41f820b6\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") "
Jul 30 17:33:21.801064 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.801025    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd67cfff-a350-42fd-ac86-da5d41f820b6-alertmanager-trusted-ca-bundle\") pod \"dd67cfff-a350-42fd-ac86-da5d41f820b6\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") "
Jul 30 17:33:21.801064 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.801047    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlwqs\" (UniqueName: \"kubernetes.io/projected/dd67cfff-a350-42fd-ac86-da5d41f820b6-kube-api-access-xlwqs\") pod \"dd67cfff-a350-42fd-ac86-da5d41f820b6\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") "
Jul 30 17:33:21.801318 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.801087    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-kube-rbac-proxy-web\") pod \"dd67cfff-a350-42fd-ac86-da5d41f820b6\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") "
Jul 30 17:33:21.801318 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.801110    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dd67cfff-a350-42fd-ac86-da5d41f820b6-alertmanager-main-db\") pod \"dd67cfff-a350-42fd-ac86-da5d41f820b6\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") "
Jul 30 17:33:21.801318 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.801128    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"dd67cfff-a350-42fd-ac86-da5d41f820b6\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") "
Jul 30 17:33:21.801318 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.801144    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dd67cfff-a350-42fd-ac86-da5d41f820b6-config-out\") pod \"dd67cfff-a350-42fd-ac86-da5d41f820b6\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") "
Jul 30 17:33:21.801318 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.801170    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd67cfff-a350-42fd-ac86-da5d41f820b6-metrics-client-ca\") pod \"dd67cfff-a350-42fd-ac86-da5d41f820b6\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") "
Jul 30 17:33:21.801574 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.801433    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd67cfff-a350-42fd-ac86-da5d41f820b6-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "dd67cfff-a350-42fd-ac86-da5d41f820b6" (UID: "dd67cfff-a350-42fd-ac86-da5d41f820b6"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:33:21.801630 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.801527    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd67cfff-a350-42fd-ac86-da5d41f820b6-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "dd67cfff-a350-42fd-ac86-da5d41f820b6" (UID: "dd67cfff-a350-42fd-ac86-da5d41f820b6"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:33:21.801684 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.801620    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-web-config\") pod \"dd67cfff-a350-42fd-ac86-da5d41f820b6\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") "
Jul 30 17:33:21.801684 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.801656    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-config-volume\") pod \"dd67cfff-a350-42fd-ac86-da5d41f820b6\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") "
Jul 30 17:33:21.801779 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.801703    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-kube-rbac-proxy\") pod \"dd67cfff-a350-42fd-ac86-da5d41f820b6\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") "
Jul 30 17:33:21.801779 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.801736    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-cluster-tls-config\") pod \"dd67cfff-a350-42fd-ac86-da5d41f820b6\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") "
Jul 30 17:33:21.801905 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.801786    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-main-tls\") pod \"dd67cfff-a350-42fd-ac86-da5d41f820b6\" (UID: \"dd67cfff-a350-42fd-ac86-da5d41f820b6\") "
Jul 30 17:33:21.802516 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.802150    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd67cfff-a350-42fd-ac86-da5d41f820b6-metrics-client-ca\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:33:21.802516 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.802254    2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd67cfff-a350-42fd-ac86-da5d41f820b6-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:33:21.803140 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.802853    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd67cfff-a350-42fd-ac86-da5d41f820b6-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "dd67cfff-a350-42fd-ac86-da5d41f820b6" (UID: "dd67cfff-a350-42fd-ac86-da5d41f820b6"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:33:21.804409 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.804064    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd67cfff-a350-42fd-ac86-da5d41f820b6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "dd67cfff-a350-42fd-ac86-da5d41f820b6" (UID: "dd67cfff-a350-42fd-ac86-da5d41f820b6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:33:21.804409 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.804136    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "dd67cfff-a350-42fd-ac86-da5d41f820b6" (UID: "dd67cfff-a350-42fd-ac86-da5d41f820b6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:33:21.804409 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.804218    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd67cfff-a350-42fd-ac86-da5d41f820b6-kube-api-access-xlwqs" (OuterVolumeSpecName: "kube-api-access-xlwqs") pod "dd67cfff-a350-42fd-ac86-da5d41f820b6" (UID: "dd67cfff-a350-42fd-ac86-da5d41f820b6"). InnerVolumeSpecName "kube-api-access-xlwqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:33:21.804648 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.804551    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "dd67cfff-a350-42fd-ac86-da5d41f820b6" (UID: "dd67cfff-a350-42fd-ac86-da5d41f820b6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:33:21.805462 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.805202    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd67cfff-a350-42fd-ac86-da5d41f820b6-config-out" (OuterVolumeSpecName: "config-out") pod "dd67cfff-a350-42fd-ac86-da5d41f820b6" (UID: "dd67cfff-a350-42fd-ac86-da5d41f820b6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:33:21.805991 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.805968    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "dd67cfff-a350-42fd-ac86-da5d41f820b6" (UID: "dd67cfff-a350-42fd-ac86-da5d41f820b6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:33:21.806210 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.806194    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "dd67cfff-a350-42fd-ac86-da5d41f820b6" (UID: "dd67cfff-a350-42fd-ac86-da5d41f820b6"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:33:21.806716 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.806696    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-config-volume" (OuterVolumeSpecName: "config-volume") pod "dd67cfff-a350-42fd-ac86-da5d41f820b6" (UID: "dd67cfff-a350-42fd-ac86-da5d41f820b6"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:33:21.810023 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.810003    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "dd67cfff-a350-42fd-ac86-da5d41f820b6" (UID: "dd67cfff-a350-42fd-ac86-da5d41f820b6"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:33:21.815559 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.815539    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-web-config" (OuterVolumeSpecName: "web-config") pod "dd67cfff-a350-42fd-ac86-da5d41f820b6" (UID: "dd67cfff-a350-42fd-ac86-da5d41f820b6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:33:21.903127 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.903094    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:33:21.903127 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.903122    2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dd67cfff-a350-42fd-ac86-da5d41f820b6-config-out\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:33:21.903127 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.903132    2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-web-config\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:33:21.903330 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.903139    2577 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:33:21.903330 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.903148    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:33:21.903330 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.903156    2577 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-cluster-tls-config\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:33:21.903330 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.903166    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-main-tls\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:33:21.903330 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.903174    2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dd67cfff-a350-42fd-ac86-da5d41f820b6-tls-assets\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:33:21.903330 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.903181    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlwqs\" (UniqueName: \"kubernetes.io/projected/dd67cfff-a350-42fd-ac86-da5d41f820b6-kube-api-access-xlwqs\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:33:21.903330 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.903196    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd67cfff-a350-42fd-ac86-da5d41f820b6-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:33:21.903330 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.903204    2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dd67cfff-a350-42fd-ac86-da5d41f820b6-alertmanager-main-db\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:33:21.923026 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:21.922996    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:33:22.517214 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.517179    2577 generic.go:356] "Generic (PLEG): container finished" podID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerID="0277ed63543e61d0d4e173864130beb20d59f7c0f63e266c645c0f4f9bfd5c19" exitCode=0
Jul 30 17:33:22.517214 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.517203    2577 generic.go:356] "Generic (PLEG): container finished" podID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerID="8d8bcfe755faf571cdba2579db49fa9404c65335bf5ea02e9d0e10816c9cfe71" exitCode=0
Jul 30 17:33:22.517693 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.517255    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dd67cfff-a350-42fd-ac86-da5d41f820b6","Type":"ContainerDied","Data":"0277ed63543e61d0d4e173864130beb20d59f7c0f63e266c645c0f4f9bfd5c19"}
Jul 30 17:33:22.517693 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.517273    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.517693 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.517321    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dd67cfff-a350-42fd-ac86-da5d41f820b6","Type":"ContainerDied","Data":"8d8bcfe755faf571cdba2579db49fa9404c65335bf5ea02e9d0e10816c9cfe71"}
Jul 30 17:33:22.517693 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.517351    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dd67cfff-a350-42fd-ac86-da5d41f820b6","Type":"ContainerDied","Data":"9b61989e1ad727216e7975eaea074fcb58db81b6feffccd9ddd3e5ed31956076"}
Jul 30 17:33:22.517693 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.517409    2577 scope.go:117] "RemoveContainer" containerID="5bed2935a830b69798c68c01347130950d6d1a942726bfc9538425080c0a3949"
Jul 30 17:33:22.525171 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.525153    2577 scope.go:117] "RemoveContainer" containerID="0277ed63543e61d0d4e173864130beb20d59f7c0f63e266c645c0f4f9bfd5c19"
Jul 30 17:33:22.532321 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.532304    2577 scope.go:117] "RemoveContainer" containerID="4b954ddf41ba2753068f01e379570aa3a53c8783f2cdfe40d039eec8f3dc9541"
Jul 30 17:33:22.539077 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.539054    2577 scope.go:117] "RemoveContainer" containerID="8d8bcfe755faf571cdba2579db49fa9404c65335bf5ea02e9d0e10816c9cfe71"
Jul 30 17:33:22.542897 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.541839    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 17:33:22.548532 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.548509    2577 scope.go:117] "RemoveContainer" containerID="652602d76fb179af3fcc9f2f0537f364f19423575369f5a83601a8596ffab4da"
Jul 30 17:33:22.549352 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.549331    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 17:33:22.555706 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.555691    2577 scope.go:117] "RemoveContainer" containerID="649514784f1c2cc0f85d4f5ea455ac3e6f7a77f2f1e4df126c7df857c315a4a7"
Jul 30 17:33:22.562679 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.562656    2577 scope.go:117] "RemoveContainer" containerID="849a5eea89be50a52d41542a56d11c093b10001a8e0b6732a2c9d9c5bb404357"
Jul 30 17:33:22.569546 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.569489    2577 scope.go:117] "RemoveContainer" containerID="5bed2935a830b69798c68c01347130950d6d1a942726bfc9538425080c0a3949"
Jul 30 17:33:22.569807 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:33:22.569783    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bed2935a830b69798c68c01347130950d6d1a942726bfc9538425080c0a3949\": container with ID starting with 5bed2935a830b69798c68c01347130950d6d1a942726bfc9538425080c0a3949 not found: ID does not exist" containerID="5bed2935a830b69798c68c01347130950d6d1a942726bfc9538425080c0a3949"
Jul 30 17:33:22.569868 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.569837    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bed2935a830b69798c68c01347130950d6d1a942726bfc9538425080c0a3949"} err="failed to get container status \"5bed2935a830b69798c68c01347130950d6d1a942726bfc9538425080c0a3949\": rpc error: code = NotFound desc = could not find container \"5bed2935a830b69798c68c01347130950d6d1a942726bfc9538425080c0a3949\": container with ID starting with 5bed2935a830b69798c68c01347130950d6d1a942726bfc9538425080c0a3949 not found: ID does not exist"
Jul 30 17:33:22.569868 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.569857    2577 scope.go:117] "RemoveContainer" containerID="0277ed63543e61d0d4e173864130beb20d59f7c0f63e266c645c0f4f9bfd5c19"
Jul 30 17:33:22.570104 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:33:22.570087    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0277ed63543e61d0d4e173864130beb20d59f7c0f63e266c645c0f4f9bfd5c19\": container with ID starting with 0277ed63543e61d0d4e173864130beb20d59f7c0f63e266c645c0f4f9bfd5c19 not found: ID does not exist" containerID="0277ed63543e61d0d4e173864130beb20d59f7c0f63e266c645c0f4f9bfd5c19"
Jul 30 17:33:22.570176 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.570114    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0277ed63543e61d0d4e173864130beb20d59f7c0f63e266c645c0f4f9bfd5c19"} err="failed to get container status \"0277ed63543e61d0d4e173864130beb20d59f7c0f63e266c645c0f4f9bfd5c19\": rpc error: code = NotFound desc = could not find container \"0277ed63543e61d0d4e173864130beb20d59f7c0f63e266c645c0f4f9bfd5c19\": container with ID starting with 0277ed63543e61d0d4e173864130beb20d59f7c0f63e266c645c0f4f9bfd5c19 not found: ID does not exist"
Jul 30 17:33:22.570176 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.570137    2577 scope.go:117] "RemoveContainer" containerID="4b954ddf41ba2753068f01e379570aa3a53c8783f2cdfe40d039eec8f3dc9541"
Jul 30 17:33:22.570366 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:33:22.570344    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b954ddf41ba2753068f01e379570aa3a53c8783f2cdfe40d039eec8f3dc9541\": container with ID starting with 4b954ddf41ba2753068f01e379570aa3a53c8783f2cdfe40d039eec8f3dc9541 not found: ID does not exist" containerID="4b954ddf41ba2753068f01e379570aa3a53c8783f2cdfe40d039eec8f3dc9541"
Jul 30 17:33:22.570429 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.570375    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b954ddf41ba2753068f01e379570aa3a53c8783f2cdfe40d039eec8f3dc9541"} err="failed to get container status \"4b954ddf41ba2753068f01e379570aa3a53c8783f2cdfe40d039eec8f3dc9541\": rpc error: code = NotFound desc = could not find container \"4b954ddf41ba2753068f01e379570aa3a53c8783f2cdfe40d039eec8f3dc9541\": container with ID starting with 4b954ddf41ba2753068f01e379570aa3a53c8783f2cdfe40d039eec8f3dc9541 not found: ID does not exist"
Jul 30 17:33:22.570429 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.570394    2577 scope.go:117] "RemoveContainer" containerID="8d8bcfe755faf571cdba2579db49fa9404c65335bf5ea02e9d0e10816c9cfe71"
Jul 30 17:33:22.570618 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:33:22.570604    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d8bcfe755faf571cdba2579db49fa9404c65335bf5ea02e9d0e10816c9cfe71\": container with ID starting with 8d8bcfe755faf571cdba2579db49fa9404c65335bf5ea02e9d0e10816c9cfe71 not found: ID does not exist" containerID="8d8bcfe755faf571cdba2579db49fa9404c65335bf5ea02e9d0e10816c9cfe71"
Jul 30 17:33:22.570695 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.570622    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d8bcfe755faf571cdba2579db49fa9404c65335bf5ea02e9d0e10816c9cfe71"} err="failed to get container status \"8d8bcfe755faf571cdba2579db49fa9404c65335bf5ea02e9d0e10816c9cfe71\": rpc error: code = NotFound desc = could not find container \"8d8bcfe755faf571cdba2579db49fa9404c65335bf5ea02e9d0e10816c9cfe71\": container with ID starting with 8d8bcfe755faf571cdba2579db49fa9404c65335bf5ea02e9d0e10816c9cfe71 not found: ID does not exist"
Jul 30 17:33:22.570695 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.570634    2577 scope.go:117] "RemoveContainer" containerID="652602d76fb179af3fcc9f2f0537f364f19423575369f5a83601a8596ffab4da"
Jul 30 17:33:22.570937 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:33:22.570922    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"652602d76fb179af3fcc9f2f0537f364f19423575369f5a83601a8596ffab4da\": container with ID starting with 652602d76fb179af3fcc9f2f0537f364f19423575369f5a83601a8596ffab4da not found: ID does not exist" containerID="652602d76fb179af3fcc9f2f0537f364f19423575369f5a83601a8596ffab4da"
Jul 30 17:33:22.571001 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.570948    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"652602d76fb179af3fcc9f2f0537f364f19423575369f5a83601a8596ffab4da"} err="failed to get container status \"652602d76fb179af3fcc9f2f0537f364f19423575369f5a83601a8596ffab4da\": rpc error: code = NotFound desc = could not find container \"652602d76fb179af3fcc9f2f0537f364f19423575369f5a83601a8596ffab4da\": container with ID starting with 652602d76fb179af3fcc9f2f0537f364f19423575369f5a83601a8596ffab4da not found: ID does not exist"
Jul 30 17:33:22.571001 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.570960    2577 scope.go:117] "RemoveContainer" containerID="649514784f1c2cc0f85d4f5ea455ac3e6f7a77f2f1e4df126c7df857c315a4a7"
Jul 30 17:33:22.571188 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:33:22.571173    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"649514784f1c2cc0f85d4f5ea455ac3e6f7a77f2f1e4df126c7df857c315a4a7\": container with ID starting with 649514784f1c2cc0f85d4f5ea455ac3e6f7a77f2f1e4df126c7df857c315a4a7 not found: ID does not exist" containerID="649514784f1c2cc0f85d4f5ea455ac3e6f7a77f2f1e4df126c7df857c315a4a7"
Jul 30 17:33:22.571227 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.571190    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"649514784f1c2cc0f85d4f5ea455ac3e6f7a77f2f1e4df126c7df857c315a4a7"} err="failed to get container status \"649514784f1c2cc0f85d4f5ea455ac3e6f7a77f2f1e4df126c7df857c315a4a7\": rpc error: code = NotFound desc = could not find container \"649514784f1c2cc0f85d4f5ea455ac3e6f7a77f2f1e4df126c7df857c315a4a7\": container with ID starting with 649514784f1c2cc0f85d4f5ea455ac3e6f7a77f2f1e4df126c7df857c315a4a7 not found: ID does not exist"
Jul 30 17:33:22.571227 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.571201    2577 scope.go:117] "RemoveContainer" containerID="849a5eea89be50a52d41542a56d11c093b10001a8e0b6732a2c9d9c5bb404357"
Jul 30 17:33:22.571399 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:33:22.571383    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"849a5eea89be50a52d41542a56d11c093b10001a8e0b6732a2c9d9c5bb404357\": container with ID starting with 849a5eea89be50a52d41542a56d11c093b10001a8e0b6732a2c9d9c5bb404357 not found: ID does not exist" containerID="849a5eea89be50a52d41542a56d11c093b10001a8e0b6732a2c9d9c5bb404357"
Jul 30 17:33:22.571441 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.571402    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"849a5eea89be50a52d41542a56d11c093b10001a8e0b6732a2c9d9c5bb404357"} err="failed to get container status \"849a5eea89be50a52d41542a56d11c093b10001a8e0b6732a2c9d9c5bb404357\": rpc error: code = NotFound desc = could not find container \"849a5eea89be50a52d41542a56d11c093b10001a8e0b6732a2c9d9c5bb404357\": container with ID starting with 849a5eea89be50a52d41542a56d11c093b10001a8e0b6732a2c9d9c5bb404357 not found: ID does not exist"
Jul 30 17:33:22.571441 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.571413    2577 scope.go:117] "RemoveContainer" containerID="5bed2935a830b69798c68c01347130950d6d1a942726bfc9538425080c0a3949"
Jul 30 17:33:22.571609 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.571590    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bed2935a830b69798c68c01347130950d6d1a942726bfc9538425080c0a3949"} err="failed to get container status \"5bed2935a830b69798c68c01347130950d6d1a942726bfc9538425080c0a3949\": rpc error: code = NotFound desc = could not find container \"5bed2935a830b69798c68c01347130950d6d1a942726bfc9538425080c0a3949\": container with ID starting with 5bed2935a830b69798c68c01347130950d6d1a942726bfc9538425080c0a3949 not found: ID does not exist"
Jul 30 17:33:22.571648 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.571612    2577 scope.go:117] "RemoveContainer" containerID="0277ed63543e61d0d4e173864130beb20d59f7c0f63e266c645c0f4f9bfd5c19"
Jul 30 17:33:22.571831 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.571797    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0277ed63543e61d0d4e173864130beb20d59f7c0f63e266c645c0f4f9bfd5c19"} err="failed to get container status \"0277ed63543e61d0d4e173864130beb20d59f7c0f63e266c645c0f4f9bfd5c19\": rpc error: code = NotFound desc = could not find container \"0277ed63543e61d0d4e173864130beb20d59f7c0f63e266c645c0f4f9bfd5c19\": container with ID starting with 0277ed63543e61d0d4e173864130beb20d59f7c0f63e266c645c0f4f9bfd5c19 not found: ID does not exist"
Jul 30 17:33:22.571878 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.571830    2577 scope.go:117] "RemoveContainer" containerID="4b954ddf41ba2753068f01e379570aa3a53c8783f2cdfe40d039eec8f3dc9541"
Jul 30 17:33:22.572051 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.572036    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b954ddf41ba2753068f01e379570aa3a53c8783f2cdfe40d039eec8f3dc9541"} err="failed to get container status \"4b954ddf41ba2753068f01e379570aa3a53c8783f2cdfe40d039eec8f3dc9541\": rpc error: code = NotFound desc = could not find container \"4b954ddf41ba2753068f01e379570aa3a53c8783f2cdfe40d039eec8f3dc9541\": container with ID starting with 4b954ddf41ba2753068f01e379570aa3a53c8783f2cdfe40d039eec8f3dc9541 not found: ID does not exist"
Jul 30 17:33:22.572087 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.572052    2577 scope.go:117] "RemoveContainer" containerID="8d8bcfe755faf571cdba2579db49fa9404c65335bf5ea02e9d0e10816c9cfe71"
Jul 30 17:33:22.572218 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.572202    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d8bcfe755faf571cdba2579db49fa9404c65335bf5ea02e9d0e10816c9cfe71"} err="failed to get container status \"8d8bcfe755faf571cdba2579db49fa9404c65335bf5ea02e9d0e10816c9cfe71\": rpc error: code = NotFound desc = could not find container \"8d8bcfe755faf571cdba2579db49fa9404c65335bf5ea02e9d0e10816c9cfe71\": container with ID starting with 8d8bcfe755faf571cdba2579db49fa9404c65335bf5ea02e9d0e10816c9cfe71 not found: ID does not exist"
Jul 30 17:33:22.572262 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.572219    2577 scope.go:117] "RemoveContainer" containerID="652602d76fb179af3fcc9f2f0537f364f19423575369f5a83601a8596ffab4da"
Jul 30 17:33:22.572465 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.572443    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"652602d76fb179af3fcc9f2f0537f364f19423575369f5a83601a8596ffab4da"} err="failed to get container status \"652602d76fb179af3fcc9f2f0537f364f19423575369f5a83601a8596ffab4da\": rpc error: code = NotFound desc = could not find container \"652602d76fb179af3fcc9f2f0537f364f19423575369f5a83601a8596ffab4da\": container with ID starting with 652602d76fb179af3fcc9f2f0537f364f19423575369f5a83601a8596ffab4da not found: ID does not exist"
Jul 30 17:33:22.572520 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.572466    2577 scope.go:117] "RemoveContainer" containerID="649514784f1c2cc0f85d4f5ea455ac3e6f7a77f2f1e4df126c7df857c315a4a7"
Jul 30 17:33:22.572616 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.572600    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"649514784f1c2cc0f85d4f5ea455ac3e6f7a77f2f1e4df126c7df857c315a4a7"} err="failed to get container status \"649514784f1c2cc0f85d4f5ea455ac3e6f7a77f2f1e4df126c7df857c315a4a7\": rpc error: code = NotFound desc = could not find container \"649514784f1c2cc0f85d4f5ea455ac3e6f7a77f2f1e4df126c7df857c315a4a7\": container with ID starting with 649514784f1c2cc0f85d4f5ea455ac3e6f7a77f2f1e4df126c7df857c315a4a7 not found: ID does not exist"
Jul 30 17:33:22.572661 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.572617    2577 scope.go:117] "RemoveContainer" containerID="849a5eea89be50a52d41542a56d11c093b10001a8e0b6732a2c9d9c5bb404357"
Jul 30 17:33:22.572853 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.572835    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"849a5eea89be50a52d41542a56d11c093b10001a8e0b6732a2c9d9c5bb404357"} err="failed to get container status \"849a5eea89be50a52d41542a56d11c093b10001a8e0b6732a2c9d9c5bb404357\": rpc error: code = NotFound desc = could not find container \"849a5eea89be50a52d41542a56d11c093b10001a8e0b6732a2c9d9c5bb404357\": container with ID starting with 849a5eea89be50a52d41542a56d11c093b10001a8e0b6732a2c9d9c5bb404357 not found: ID does not exist"
Jul 30 17:33:22.581504 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.581479    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 17:33:22.581712 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.581701    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="config-reloader"
Jul 30 17:33:22.581756 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.581714    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="config-reloader"
Jul 30 17:33:22.581756 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.581726    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="kube-rbac-proxy-web"
Jul 30 17:33:22.581756 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.581732    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="kube-rbac-proxy-web"
Jul 30 17:33:22.581756 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.581745    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="kube-rbac-proxy-metric"
Jul 30 17:33:22.581756 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.581752    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="kube-rbac-proxy-metric"
Jul 30 17:33:22.581923 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.581759    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="alertmanager"
Jul 30 17:33:22.581923 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.581765    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="alertmanager"
Jul 30 17:33:22.581923 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.581771    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="prom-label-proxy"
Jul 30 17:33:22.581923 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.581776    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="prom-label-proxy"
Jul 30 17:33:22.581923 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.581785    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="init-config-reloader"
Jul 30 17:33:22.581923 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.581790    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="init-config-reloader"
Jul 30 17:33:22.581923 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.581795    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="kube-rbac-proxy"
Jul 30 17:33:22.581923 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.581799    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="kube-rbac-proxy"
Jul 30 17:33:22.581923 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.581857    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="kube-rbac-proxy"
Jul 30 17:33:22.581923 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.581866    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="kube-rbac-proxy-web"
Jul 30 17:33:22.581923 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.581873    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="kube-rbac-proxy-metric"
Jul 30 17:33:22.581923 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.581880    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="prom-label-proxy"
Jul 30 17:33:22.581923 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.581886    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="alertmanager"
Jul 30 17:33:22.581923 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.581892    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" containerName="config-reloader"
Jul 30 17:33:22.585680 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.585664    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.587782 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.587755    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 30 17:33:22.588224 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.588207    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 30 17:33:22.588331 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.588222    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rtw87\""
Jul 30 17:33:22.588331 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.588209    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 30 17:33:22.588331 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.588210    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 30 17:33:22.588331 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.588210    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 30 17:33:22.588536 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.588381    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 30 17:33:22.588621 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.588606    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 30 17:33:22.588674 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.588624    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 30 17:33:22.594585 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.594566    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 30 17:33:22.603804 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.603776    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 17:33:22.608691 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.608671    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-web-config\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.608789 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.608714    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.608848 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.608776    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.608848 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.608834    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.608927 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.608868    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.608927 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.608896    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-config-volume\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.608991 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.608923    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.608991 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.608963    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.609066 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.608986    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.609098 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.609074    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-config-out\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.609130 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.609105    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.609160 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.609146    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrhlh\" (UniqueName: \"kubernetes.io/projected/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-kube-api-access-nrhlh\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.609194 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.609166    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.710530 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.710499    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-config-out\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.710530 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.710528    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.710768 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.710554    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrhlh\" (UniqueName: \"kubernetes.io/projected/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-kube-api-access-nrhlh\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.710768 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.710645    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.711160 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.710834    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-web-config\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.711160 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.710877    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.711160 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.710919    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.711160 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.710945    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.711160 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.710996    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.711160 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.711013    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.711160 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.711029    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-config-volume\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.711464 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.711342    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.711464 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.711365    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.711464 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.711404    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.711464 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.711446    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.712490 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.712445    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.714399 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.714230    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-config-out\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.714399 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.714252    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.714575 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.714520    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-config-volume\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.714640 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.714600    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.714691 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.714632    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.714765 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.714747    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.714934 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.714917    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.715251 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.715237    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.715762 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.715746    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-web-config\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.720483 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.720464    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrhlh\" (UniqueName: \"kubernetes.io/projected/982e116f-d739-42f4-bfe6-cbb4f5ca6d1f-kube-api-access-nrhlh\") pod \"alertmanager-main-0\" (UID: \"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:22.895030 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:22.894941    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 17:33:23.044876 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:23.044764    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 17:33:23.047601 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:33:23.047565    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod982e116f_d739_42f4_bfe6_cbb4f5ca6d1f.slice/crio-e81244e98189a269b08cd98e3b346a008b3f63897f62a9a730da55e7f4a29e29 WatchSource:0}: Error finding container e81244e98189a269b08cd98e3b346a008b3f63897f62a9a730da55e7f4a29e29: Status 404 returned error can't find the container with id e81244e98189a269b08cd98e3b346a008b3f63897f62a9a730da55e7f4a29e29
Jul 30 17:33:23.521728 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:23.521688    2577 generic.go:356] "Generic (PLEG): container finished" podID="982e116f-d739-42f4-bfe6-cbb4f5ca6d1f" containerID="cee936a86769ef5032eef2ba741b327235c8b3f464a3668f41a0ce21c6e67bff" exitCode=0
Jul 30 17:33:23.522335 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:23.521786    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f","Type":"ContainerDied","Data":"cee936a86769ef5032eef2ba741b327235c8b3f464a3668f41a0ce21c6e67bff"}
Jul 30 17:33:23.522335 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:23.521913    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f","Type":"ContainerStarted","Data":"e81244e98189a269b08cd98e3b346a008b3f63897f62a9a730da55e7f4a29e29"}
Jul 30 17:33:24.108802 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.108715    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd67cfff-a350-42fd-ac86-da5d41f820b6" path="/var/lib/kubelet/pods/dd67cfff-a350-42fd-ac86-da5d41f820b6/volumes"
Jul 30 17:33:24.519274 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.519244    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"]
Jul 30 17:33:24.521999 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.521984    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.524244 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.524196    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 30 17:33:24.524412 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.524269    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 30 17:33:24.524603 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.524582    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 30 17:33:24.524696 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.524622    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 30 17:33:24.524696 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.524628    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 30 17:33:24.524696 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.524689    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-9llzr\""
Jul 30 17:33:24.530899 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.530880    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 30 17:33:24.532073 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.532051    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f","Type":"ContainerStarted","Data":"9d63ac4200dfbaf5008882f02f0868ab3e71ca6b8ecc48dc624d057ec903fdf3"}
Jul 30 17:33:24.532150 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.532128    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f","Type":"ContainerStarted","Data":"5ca7e4fb9ee783b4c451d6b198132daff32246a2bbf560583b5c516535f405b5"}
Jul 30 17:33:24.532193 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.532172    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f","Type":"ContainerStarted","Data":"feae71e49c78144a30f9385a40ec1f4c129850351da8d20828110aef3ac0b315"}
Jul 30 17:33:24.532228 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.532218    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f","Type":"ContainerStarted","Data":"35a06aaa5cda8db5634ca5fe57594df3e0bb5dbfd736882d28a885b7562a82ac"}
Jul 30 17:33:24.532270 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.532258    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f","Type":"ContainerStarted","Data":"a994db8ac1201b408d77cac5b5c6f617e54e43b649cdce048307c939eb4ce734"}
Jul 30 17:33:24.532316 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.532305    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"982e116f-d739-42f4-bfe6-cbb4f5ca6d1f","Type":"ContainerStarted","Data":"f55fa2d85b08a965eb3128f13ad61c8ad85d51ae4b6eb42760874096615881cd"}
Jul 30 17:33:24.539138 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.539119    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"]
Jul 30 17:33:24.583119 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.583061    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.567292906 podStartE2EDuration="2.58304556s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.752654ms" isStatefulPod=false podCreationTimestamp="2026-07-30 17:33:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:33:24.581417378 +0000 UTC m=+379.034981371" watchObservedRunningTime="2026-07-30 17:33:24.58304556 +0000 UTC m=+379.036609551"
Jul 30 17:33:24.626380 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.626350    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bb04c41e-aa36-409e-93e8-0e9abf1a764d-federate-client-tls\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.626570 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.626479    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bb04c41e-aa36-409e-93e8-0e9abf1a764d-telemeter-client-tls\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.626873 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.626798    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb04c41e-aa36-409e-93e8-0e9abf1a764d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.627021 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.626908    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb04c41e-aa36-409e-93e8-0e9abf1a764d-metrics-client-ca\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.627021 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.626931    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb04c41e-aa36-409e-93e8-0e9abf1a764d-serving-certs-ca-bundle\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.627021 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.626949    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bb04c41e-aa36-409e-93e8-0e9abf1a764d-secret-telemeter-client\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.627021 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.626963    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47mhp\" (UniqueName: \"kubernetes.io/projected/bb04c41e-aa36-409e-93e8-0e9abf1a764d-kube-api-access-47mhp\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.627021 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.627004    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bb04c41e-aa36-409e-93e8-0e9abf1a764d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.727720 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.727674    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bb04c41e-aa36-409e-93e8-0e9abf1a764d-federate-client-tls\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.727911 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.727742    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bb04c41e-aa36-409e-93e8-0e9abf1a764d-telemeter-client-tls\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.727911 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.727794    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb04c41e-aa36-409e-93e8-0e9abf1a764d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.727985 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.727938    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb04c41e-aa36-409e-93e8-0e9abf1a764d-metrics-client-ca\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.727985 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.727974    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb04c41e-aa36-409e-93e8-0e9abf1a764d-serving-certs-ca-bundle\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.728079 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.728005    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bb04c41e-aa36-409e-93e8-0e9abf1a764d-secret-telemeter-client\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.728079 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.728028    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47mhp\" (UniqueName: \"kubernetes.io/projected/bb04c41e-aa36-409e-93e8-0e9abf1a764d-kube-api-access-47mhp\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.728079 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.728061    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bb04c41e-aa36-409e-93e8-0e9abf1a764d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.728926 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.728900    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb04c41e-aa36-409e-93e8-0e9abf1a764d-serving-certs-ca-bundle\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.728926 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.728919    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb04c41e-aa36-409e-93e8-0e9abf1a764d-metrics-client-ca\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.729115 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.729039    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb04c41e-aa36-409e-93e8-0e9abf1a764d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.730431 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.730408    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/bb04c41e-aa36-409e-93e8-0e9abf1a764d-telemeter-client-tls\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.730980 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.730945    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bb04c41e-aa36-409e-93e8-0e9abf1a764d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.731075 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.731030    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/bb04c41e-aa36-409e-93e8-0e9abf1a764d-secret-telemeter-client\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.731139 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.731113    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/bb04c41e-aa36-409e-93e8-0e9abf1a764d-federate-client-tls\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.737311 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.737291    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47mhp\" (UniqueName: \"kubernetes.io/projected/bb04c41e-aa36-409e-93e8-0e9abf1a764d-kube-api-access-47mhp\") pod \"telemeter-client-566b6bf4dd-r8c5m\" (UID: \"bb04c41e-aa36-409e-93e8-0e9abf1a764d\") " pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.833843 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.833735    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"
Jul 30 17:33:24.983890 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:24.983859    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m"]
Jul 30 17:33:24.987387 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:33:24.987361    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb04c41e_aa36_409e_93e8_0e9abf1a764d.slice/crio-3b960e55fb9749b52ac25139c8b690f4ae6618601cb1b9e7e39b0fc5b8c4251e WatchSource:0}: Error finding container 3b960e55fb9749b52ac25139c8b690f4ae6618601cb1b9e7e39b0fc5b8c4251e: Status 404 returned error can't find the container with id 3b960e55fb9749b52ac25139c8b690f4ae6618601cb1b9e7e39b0fc5b8c4251e
Jul 30 17:33:25.536846 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:25.536782    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m" event={"ID":"bb04c41e-aa36-409e-93e8-0e9abf1a764d","Type":"ContainerStarted","Data":"3b960e55fb9749b52ac25139c8b690f4ae6618601cb1b9e7e39b0fc5b8c4251e"}
Jul 30 17:33:27.547778 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:27.547745    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m" event={"ID":"bb04c41e-aa36-409e-93e8-0e9abf1a764d","Type":"ContainerStarted","Data":"b0a9ad2fa267496270e22f452e820e9c46f97d74e1beb17164eff2590cb3f195"}
Jul 30 17:33:27.548175 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:27.547810    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m" event={"ID":"bb04c41e-aa36-409e-93e8-0e9abf1a764d","Type":"ContainerStarted","Data":"6e0674e1b77544e9fb0ea183ddbf8be7114c70cedb3ef1702f5c5ca5452d21ec"}
Jul 30 17:33:27.548175 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:27.547861    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m" event={"ID":"bb04c41e-aa36-409e-93e8-0e9abf1a764d","Type":"ContainerStarted","Data":"dc5103d186527bcdaacc6a141a99143bfd558b40cf32436c64c878b0f6d2cfb0"}
Jul 30 17:33:27.573861 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:27.573743    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-566b6bf4dd-r8c5m" podStartSLOduration=1.5080304249999998 podStartE2EDuration="3.573726916s" totalImagesPullingTime="2.065696491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:33:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:33:27.572466968 +0000 UTC m=+382.026030975" watchObservedRunningTime="2026-07-30 17:33:27.573726916 +0000 UTC m=+382.027290907"
Jul 30 17:33:28.638299 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:28.638261    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:33:28.735277 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:28.735240    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:33:35.222926 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:35.222891    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:33:35.320617 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:35.320582    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:33:39.157166 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:39.157133    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d97744b78-gnthl"]
Jul 30 17:33:39.282518 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:39.282483    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:33:39.384600 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:39.384569    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:33:39.445031 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:33:39.444960    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:34:04.176281 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.176243    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6d97744b78-gnthl" podUID="e971b04e-381f-4bed-85a4-6ea1fa1c52b5" containerName="console" containerID="cri-o://3cd6e666affae65be4bc2604b687da56ed880256fd888a3d51b89843b427d9cd" gracePeriod=15
Jul 30 17:34:04.413490 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.413466    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:34:04.554563 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.554523    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-trusted-ca-bundle\") pod \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") "
Jul 30 17:34:04.554563 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.554566    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-console-config\") pod \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") "
Jul 30 17:34:04.554903 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.554593    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-service-ca\") pod \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") "
Jul 30 17:34:04.554903 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.554623    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-console-oauth-config\") pod \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") "
Jul 30 17:34:04.554903 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.554653    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwphz\" (UniqueName: \"kubernetes.io/projected/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-kube-api-access-qwphz\") pod \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") "
Jul 30 17:34:04.554903 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.554699    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-oauth-serving-cert\") pod \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") "
Jul 30 17:34:04.554903 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.554748    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-console-serving-cert\") pod \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\" (UID: \"e971b04e-381f-4bed-85a4-6ea1fa1c52b5\") "
Jul 30 17:34:04.555164 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.555123    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-service-ca" (OuterVolumeSpecName: "service-ca") pod "e971b04e-381f-4bed-85a4-6ea1fa1c52b5" (UID: "e971b04e-381f-4bed-85a4-6ea1fa1c52b5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:34:04.555164 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.555143    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e971b04e-381f-4bed-85a4-6ea1fa1c52b5" (UID: "e971b04e-381f-4bed-85a4-6ea1fa1c52b5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:34:04.555270 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.555162    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-console-config" (OuterVolumeSpecName: "console-config") pod "e971b04e-381f-4bed-85a4-6ea1fa1c52b5" (UID: "e971b04e-381f-4bed-85a4-6ea1fa1c52b5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:34:04.555270 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.555208    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e971b04e-381f-4bed-85a4-6ea1fa1c52b5" (UID: "e971b04e-381f-4bed-85a4-6ea1fa1c52b5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:34:04.557138 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.557112    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e971b04e-381f-4bed-85a4-6ea1fa1c52b5" (UID: "e971b04e-381f-4bed-85a4-6ea1fa1c52b5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:34:04.557234 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.557136    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e971b04e-381f-4bed-85a4-6ea1fa1c52b5" (UID: "e971b04e-381f-4bed-85a4-6ea1fa1c52b5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:34:04.557234 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.557165    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-kube-api-access-qwphz" (OuterVolumeSpecName: "kube-api-access-qwphz") pod "e971b04e-381f-4bed-85a4-6ea1fa1c52b5" (UID: "e971b04e-381f-4bed-85a4-6ea1fa1c52b5"). InnerVolumeSpecName "kube-api-access-qwphz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:34:04.656128 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.656089    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-trusted-ca-bundle\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:34:04.656128 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.656120    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-console-config\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:34:04.656128 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.656133    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-service-ca\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:34:04.656370 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.656144    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-console-oauth-config\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:34:04.656370 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.656156    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwphz\" (UniqueName: \"kubernetes.io/projected/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-kube-api-access-qwphz\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:34:04.656370 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.656167    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-oauth-serving-cert\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:34:04.656370 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.656178    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e971b04e-381f-4bed-85a4-6ea1fa1c52b5-console-serving-cert\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:34:04.692682 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.692647    2577 generic.go:356] "Generic (PLEG): container finished" podID="e971b04e-381f-4bed-85a4-6ea1fa1c52b5" containerID="3cd6e666affae65be4bc2604b687da56ed880256fd888a3d51b89843b427d9cd" exitCode=0
Jul 30 17:34:04.692878 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.692691    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d97744b78-gnthl" event={"ID":"e971b04e-381f-4bed-85a4-6ea1fa1c52b5","Type":"ContainerDied","Data":"3cd6e666affae65be4bc2604b687da56ed880256fd888a3d51b89843b427d9cd"}
Jul 30 17:34:04.692878 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.692695    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d97744b78-gnthl"
Jul 30 17:34:04.692878 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.692731    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d97744b78-gnthl" event={"ID":"e971b04e-381f-4bed-85a4-6ea1fa1c52b5","Type":"ContainerDied","Data":"a7f68151fb32e5751aba3a5c8a41e7791068ddfae7bc8c38e17919cffc909b70"}
Jul 30 17:34:04.692878 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.692750    2577 scope.go:117] "RemoveContainer" containerID="3cd6e666affae65be4bc2604b687da56ed880256fd888a3d51b89843b427d9cd"
Jul 30 17:34:04.701358 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.701341    2577 scope.go:117] "RemoveContainer" containerID="3cd6e666affae65be4bc2604b687da56ed880256fd888a3d51b89843b427d9cd"
Jul 30 17:34:04.701631 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:34:04.701614    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cd6e666affae65be4bc2604b687da56ed880256fd888a3d51b89843b427d9cd\": container with ID starting with 3cd6e666affae65be4bc2604b687da56ed880256fd888a3d51b89843b427d9cd not found: ID does not exist" containerID="3cd6e666affae65be4bc2604b687da56ed880256fd888a3d51b89843b427d9cd"
Jul 30 17:34:04.701668 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.701640    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cd6e666affae65be4bc2604b687da56ed880256fd888a3d51b89843b427d9cd"} err="failed to get container status \"3cd6e666affae65be4bc2604b687da56ed880256fd888a3d51b89843b427d9cd\": rpc error: code = NotFound desc = could not find container \"3cd6e666affae65be4bc2604b687da56ed880256fd888a3d51b89843b427d9cd\": container with ID starting with 3cd6e666affae65be4bc2604b687da56ed880256fd888a3d51b89843b427d9cd not found: ID does not exist"
Jul 30 17:34:04.714002 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.713981    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d97744b78-gnthl"]
Jul 30 17:34:04.717210 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:04.717190    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d97744b78-gnthl"]
Jul 30 17:34:06.109388 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:06.109356    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e971b04e-381f-4bed-85a4-6ea1fa1c52b5" path="/var/lib/kubelet/pods/e971b04e-381f-4bed-85a4-6ea1fa1c52b5/volumes"
Jul 30 17:34:07.716406 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:07.716374    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:34:07.771922 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:07.771896    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:34:30.461721 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:30.461677    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:34:34.660410 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:34.660378    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:34:34.719961 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:34.719938    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:34:37.543204 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:37.543172    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:34:37.644191 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:37.644157    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:34:46.650180 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:46.650148    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:34:46.707984 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:46.707954    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:34:48.563469 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:48.563432    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:34:48.668374 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:48.668345    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:34:48.767412 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:48.767376    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:34:57.788250 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:57.788214    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:34:57.886333 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:57.886302    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:34:58.106379 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:34:58.106302    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:35:34.235389 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:35:34.235353    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:35:34.335297 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:35:34.335267    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:06.307074 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:06.306985    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:07.177918 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:07.177884    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:09.920290 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:09.920251    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:10.035174 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:10.035143    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:11.378037 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:11.378004    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:11.472981 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:11.472945    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:12.065494 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:12.065464    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:12.172381 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:12.172352    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:13.568697 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:13.568664    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:13.745887 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:13.745840    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:16.708598 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:16.708570    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:16.771798 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:16.771762    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:19.841171 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:19.841083    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:19.910495 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:19.910466    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:21.951727 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:21.951696    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:22.023996 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:22.023962    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4tbzv"]
Jul 30 17:36:22.024244 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:22.024232    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e971b04e-381f-4bed-85a4-6ea1fa1c52b5" containerName="console"
Jul 30 17:36:22.024285 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:22.024246    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e971b04e-381f-4bed-85a4-6ea1fa1c52b5" containerName="console"
Jul 30 17:36:22.024321 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:22.024302    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e971b04e-381f-4bed-85a4-6ea1fa1c52b5" containerName="console"
Jul 30 17:36:22.027083 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:22.027068    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-4tbzv"
Jul 30 17:36:22.029147 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:22.029124    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-hkq2z\""
Jul 30 17:36:22.029376 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:22.029362    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 30 17:36:22.030085 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:22.030065    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 30 17:36:22.037727 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:22.037703    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4tbzv"]
Jul 30 17:36:22.069331 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:22.069298    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bdf69248-af4b-4040-b367-62c309785084-bound-sa-token\") pod \"cert-manager-55b4d954d5-4tbzv\" (UID: \"bdf69248-af4b-4040-b367-62c309785084\") " pod="cert-manager/cert-manager-55b4d954d5-4tbzv"
Jul 30 17:36:22.069490 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:22.069388    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmxbx\" (UniqueName: \"kubernetes.io/projected/bdf69248-af4b-4040-b367-62c309785084-kube-api-access-wmxbx\") pod \"cert-manager-55b4d954d5-4tbzv\" (UID: \"bdf69248-af4b-4040-b367-62c309785084\") " pod="cert-manager/cert-manager-55b4d954d5-4tbzv"
Jul 30 17:36:22.170674 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:22.170640    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmxbx\" (UniqueName: \"kubernetes.io/projected/bdf69248-af4b-4040-b367-62c309785084-kube-api-access-wmxbx\") pod \"cert-manager-55b4d954d5-4tbzv\" (UID: \"bdf69248-af4b-4040-b367-62c309785084\") " pod="cert-manager/cert-manager-55b4d954d5-4tbzv"
Jul 30 17:36:22.170674 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:22.170690    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bdf69248-af4b-4040-b367-62c309785084-bound-sa-token\") pod \"cert-manager-55b4d954d5-4tbzv\" (UID: \"bdf69248-af4b-4040-b367-62c309785084\") " pod="cert-manager/cert-manager-55b4d954d5-4tbzv"
Jul 30 17:36:22.179490 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:22.179463    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bdf69248-af4b-4040-b367-62c309785084-bound-sa-token\") pod \"cert-manager-55b4d954d5-4tbzv\" (UID: \"bdf69248-af4b-4040-b367-62c309785084\") " pod="cert-manager/cert-manager-55b4d954d5-4tbzv"
Jul 30 17:36:22.179671 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:22.179651    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmxbx\" (UniqueName: \"kubernetes.io/projected/bdf69248-af4b-4040-b367-62c309785084-kube-api-access-wmxbx\") pod \"cert-manager-55b4d954d5-4tbzv\" (UID: \"bdf69248-af4b-4040-b367-62c309785084\") " pod="cert-manager/cert-manager-55b4d954d5-4tbzv"
Jul 30 17:36:22.335959 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:22.335929    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-4tbzv"
Jul 30 17:36:22.462916 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:22.462886    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4tbzv"]
Jul 30 17:36:22.465236 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:36:22.465202    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbdf69248_af4b_4040_b367_62c309785084.slice/crio-c2deadd6eba4885cefbf46dc174428c3177942c603cdf6a4d28a0986a101ae5f WatchSource:0}: Error finding container c2deadd6eba4885cefbf46dc174428c3177942c603cdf6a4d28a0986a101ae5f: Status 404 returned error can't find the container with id c2deadd6eba4885cefbf46dc174428c3177942c603cdf6a4d28a0986a101ae5f
Jul 30 17:36:23.183972 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:23.183940    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-4tbzv" event={"ID":"bdf69248-af4b-4040-b367-62c309785084","Type":"ContainerStarted","Data":"c2deadd6eba4885cefbf46dc174428c3177942c603cdf6a4d28a0986a101ae5f"}
Jul 30 17:36:28.204810 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:28.204769    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-4tbzv" event={"ID":"bdf69248-af4b-4040-b367-62c309785084","Type":"ContainerStarted","Data":"30e7258494d5fe9bb6acff9dd9af918d342b35685088fa41fa20051d69644203"}
Jul 30 17:36:28.220647 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:28.220596    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-4tbzv" podStartSLOduration=1.469881968 podStartE2EDuration="6.220581998s" totalImagesPullingTime="4.75070003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:36:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:36:28.220352641 +0000 UTC m=+562.673916631" watchObservedRunningTime="2026-07-30 17:36:28.220581998 +0000 UTC m=+562.674145988"
Jul 30 17:36:28.321365 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:28.321324    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:28.421326 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:28.421294    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:34.654450 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:34.654420    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:34.755057 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:34.755032    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:36.609255 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:36.609219    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:48.536242 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:48.536206    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:48.632159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:48.632117    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:49.580859 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.580829    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c"]
Jul 30 17:36:49.583998 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.583979    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c"
Jul 30 17:36:49.586174 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.586154    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 30 17:36:49.586353 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.586335    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 30 17:36:49.586692 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.586677    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 30 17:36:49.586777 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.586762    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-thdx9\""
Jul 30 17:36:49.587806 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.587789    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 30 17:36:49.599638 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.599619    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c"]
Jul 30 17:36:49.704039 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.704005    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9847f8a2-f792-433a-8e4e-25cad18e13d2-webhook-cert\") pod \"opendatahub-operator-controller-manager-975dfc96f-6pz5c\" (UID: \"9847f8a2-f792-433a-8e4e-25cad18e13d2\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c"
Jul 30 17:36:49.704207 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.704068    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9847f8a2-f792-433a-8e4e-25cad18e13d2-manifests\") pod \"opendatahub-operator-controller-manager-975dfc96f-6pz5c\" (UID: \"9847f8a2-f792-433a-8e4e-25cad18e13d2\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c"
Jul 30 17:36:49.704207 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.704108    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnn7w\" (UniqueName: \"kubernetes.io/projected/9847f8a2-f792-433a-8e4e-25cad18e13d2-kube-api-access-pnn7w\") pod \"opendatahub-operator-controller-manager-975dfc96f-6pz5c\" (UID: \"9847f8a2-f792-433a-8e4e-25cad18e13d2\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c"
Jul 30 17:36:49.704207 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.704138    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9847f8a2-f792-433a-8e4e-25cad18e13d2-tmp\") pod \"opendatahub-operator-controller-manager-975dfc96f-6pz5c\" (UID: \"9847f8a2-f792-433a-8e4e-25cad18e13d2\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c"
Jul 30 17:36:49.704310 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.704209    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9847f8a2-f792-433a-8e4e-25cad18e13d2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-975dfc96f-6pz5c\" (UID: \"9847f8a2-f792-433a-8e4e-25cad18e13d2\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c"
Jul 30 17:36:49.804998 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.804968    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9847f8a2-f792-433a-8e4e-25cad18e13d2-manifests\") pod \"opendatahub-operator-controller-manager-975dfc96f-6pz5c\" (UID: \"9847f8a2-f792-433a-8e4e-25cad18e13d2\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c"
Jul 30 17:36:49.805164 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.805007    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnn7w\" (UniqueName: \"kubernetes.io/projected/9847f8a2-f792-433a-8e4e-25cad18e13d2-kube-api-access-pnn7w\") pod \"opendatahub-operator-controller-manager-975dfc96f-6pz5c\" (UID: \"9847f8a2-f792-433a-8e4e-25cad18e13d2\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c"
Jul 30 17:36:49.805164 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.805026    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9847f8a2-f792-433a-8e4e-25cad18e13d2-tmp\") pod \"opendatahub-operator-controller-manager-975dfc96f-6pz5c\" (UID: \"9847f8a2-f792-433a-8e4e-25cad18e13d2\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c"
Jul 30 17:36:49.805164 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.805053    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9847f8a2-f792-433a-8e4e-25cad18e13d2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-975dfc96f-6pz5c\" (UID: \"9847f8a2-f792-433a-8e4e-25cad18e13d2\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c"
Jul 30 17:36:49.805164 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.805097    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9847f8a2-f792-433a-8e4e-25cad18e13d2-webhook-cert\") pod \"opendatahub-operator-controller-manager-975dfc96f-6pz5c\" (UID: \"9847f8a2-f792-433a-8e4e-25cad18e13d2\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c"
Jul 30 17:36:49.805450 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.805424    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9847f8a2-f792-433a-8e4e-25cad18e13d2-manifests\") pod \"opendatahub-operator-controller-manager-975dfc96f-6pz5c\" (UID: \"9847f8a2-f792-433a-8e4e-25cad18e13d2\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c"
Jul 30 17:36:49.807525 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.807486    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9847f8a2-f792-433a-8e4e-25cad18e13d2-tmp\") pod \"opendatahub-operator-controller-manager-975dfc96f-6pz5c\" (UID: \"9847f8a2-f792-433a-8e4e-25cad18e13d2\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c"
Jul 30 17:36:49.807617 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.807558    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9847f8a2-f792-433a-8e4e-25cad18e13d2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-975dfc96f-6pz5c\" (UID: \"9847f8a2-f792-433a-8e4e-25cad18e13d2\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c"
Jul 30 17:36:49.807668 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.807637    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9847f8a2-f792-433a-8e4e-25cad18e13d2-webhook-cert\") pod \"opendatahub-operator-controller-manager-975dfc96f-6pz5c\" (UID: \"9847f8a2-f792-433a-8e4e-25cad18e13d2\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c"
Jul 30 17:36:49.822743 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.822722    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnn7w\" (UniqueName: \"kubernetes.io/projected/9847f8a2-f792-433a-8e4e-25cad18e13d2-kube-api-access-pnn7w\") pod \"opendatahub-operator-controller-manager-975dfc96f-6pz5c\" (UID: \"9847f8a2-f792-433a-8e4e-25cad18e13d2\") " pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c"
Jul 30 17:36:49.893763 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.893680    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c"
Jul 30 17:36:49.951026 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:49.950995    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:50.041291 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:50.041270    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c"]
Jul 30 17:36:50.043413 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:36:50.043386    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9847f8a2_f792_433a_8e4e_25cad18e13d2.slice/crio-96eb90b0eb0a731c5cebce95ffa3a02c395ab09f13fbba77bd913bccd75a65a6 WatchSource:0}: Error finding container 96eb90b0eb0a731c5cebce95ffa3a02c395ab09f13fbba77bd913bccd75a65a6: Status 404 returned error can't find the container with id 96eb90b0eb0a731c5cebce95ffa3a02c395ab09f13fbba77bd913bccd75a65a6
Jul 30 17:36:50.043707 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:50.043680    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:50.116021 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:50.115987    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:50.218368 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:50.218292    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:50.286173 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:50.286139    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c" event={"ID":"9847f8a2-f792-433a-8e4e-25cad18e13d2","Type":"ContainerStarted","Data":"96eb90b0eb0a731c5cebce95ffa3a02c395ab09f13fbba77bd913bccd75a65a6"}
Jul 30 17:36:50.316142 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:50.316116    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:50.846941 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:50.846770    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:50.948031 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:50.948007    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:36:53.306561 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:53.306523    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c" event={"ID":"9847f8a2-f792-433a-8e4e-25cad18e13d2","Type":"ContainerStarted","Data":"b3e9ba6c45e1536efd6996f432da1bc9dd526d7930b80537218d8dce7d739041"}
Jul 30 17:36:54.311924 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:54.311835    2577 generic.go:356] "Generic (PLEG): container finished" podID="9847f8a2-f792-433a-8e4e-25cad18e13d2" containerID="b3e9ba6c45e1536efd6996f432da1bc9dd526d7930b80537218d8dce7d739041" exitCode=0
Jul 30 17:36:54.311924 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:54.311887    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c" event={"ID":"9847f8a2-f792-433a-8e4e-25cad18e13d2","Type":"ContainerDied","Data":"b3e9ba6c45e1536efd6996f432da1bc9dd526d7930b80537218d8dce7d739041"}
Jul 30 17:36:55.317870 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:55.317805    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c" event={"ID":"9847f8a2-f792-433a-8e4e-25cad18e13d2","Type":"ContainerStarted","Data":"202505d953d1370d9defaad1671abaf1dd4ef798d9dd7063722b804da90a8b11"}
Jul 30 17:36:55.318278 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:55.317935    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c"
Jul 30 17:36:55.341716 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:55.341665    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c" podStartSLOduration=1.990429782 podStartE2EDuration="6.341650747s" totalImagesPullingTime="3.33914649s" totalInitContainerRuntime="1.012074475s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:36:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:36:55.340415892 +0000 UTC m=+589.793979882" watchObservedRunningTime="2026-07-30 17:36:55.341650747 +0000 UTC m=+589.795214738"
Jul 30 17:36:56.338625 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:36:56.338590    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:00.310283 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:00.310252    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:00.719498 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:00.719414    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:00.779744 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:00.779722    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:06.324291 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:06.324262    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-975dfc96f-6pz5c"
Jul 30 17:37:08.650320 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:08.650291    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:08.821141 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:08.821114    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:09.275430 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.275399    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:09.493503 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.493463    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6ff87f57bc-md8k2"]
Jul 30 17:37:09.496328 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.496307    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6ff87f57bc-md8k2"
Jul 30 17:37:09.500493 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.500468    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 30 17:37:09.500636 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.500467    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 30 17:37:09.501025 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.501009    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 30 17:37:09.501274 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.501262    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 30 17:37:09.503568 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.503508    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-89m26\""
Jul 30 17:37:09.515861 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.515808    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6ff87f57bc-md8k2"]
Jul 30 17:37:09.577406 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.577305    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19d2cd9d-54a4-44d9-b08c-45c3afa59146-tmp\") pod \"kube-auth-proxy-6ff87f57bc-md8k2\" (UID: \"19d2cd9d-54a4-44d9-b08c-45c3afa59146\") " pod="openshift-ingress/kube-auth-proxy-6ff87f57bc-md8k2"
Jul 30 17:37:09.577406 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.577347    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19d2cd9d-54a4-44d9-b08c-45c3afa59146-tls-certs\") pod \"kube-auth-proxy-6ff87f57bc-md8k2\" (UID: \"19d2cd9d-54a4-44d9-b08c-45c3afa59146\") " pod="openshift-ingress/kube-auth-proxy-6ff87f57bc-md8k2"
Jul 30 17:37:09.577598 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.577453    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phznb\" (UniqueName: \"kubernetes.io/projected/19d2cd9d-54a4-44d9-b08c-45c3afa59146-kube-api-access-phznb\") pod \"kube-auth-proxy-6ff87f57bc-md8k2\" (UID: \"19d2cd9d-54a4-44d9-b08c-45c3afa59146\") " pod="openshift-ingress/kube-auth-proxy-6ff87f57bc-md8k2"
Jul 30 17:37:09.625492 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.625453    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jtrxh"]
Jul 30 17:37:09.628009 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.627988    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-jtrxh"
Jul 30 17:37:09.631010 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.630985    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 30 17:37:09.631320 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.631302    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-7d9g4\""
Jul 30 17:37:09.638749 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.638727    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jtrxh"]
Jul 30 17:37:09.678404 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.678361    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkjgx\" (UniqueName: \"kubernetes.io/projected/29a8ca15-2df6-4ba4-adfc-a0024f4c3648-kube-api-access-pkjgx\") pod \"odh-model-controller-7998c67dc4-jtrxh\" (UID: \"29a8ca15-2df6-4ba4-adfc-a0024f4c3648\") " pod="opendatahub/odh-model-controller-7998c67dc4-jtrxh"
Jul 30 17:37:09.678844 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.678480    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phznb\" (UniqueName: \"kubernetes.io/projected/19d2cd9d-54a4-44d9-b08c-45c3afa59146-kube-api-access-phznb\") pod \"kube-auth-proxy-6ff87f57bc-md8k2\" (UID: \"19d2cd9d-54a4-44d9-b08c-45c3afa59146\") " pod="openshift-ingress/kube-auth-proxy-6ff87f57bc-md8k2"
Jul 30 17:37:09.678844 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.678534    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29a8ca15-2df6-4ba4-adfc-a0024f4c3648-cert\") pod \"odh-model-controller-7998c67dc4-jtrxh\" (UID: \"29a8ca15-2df6-4ba4-adfc-a0024f4c3648\") " pod="opendatahub/odh-model-controller-7998c67dc4-jtrxh"
Jul 30 17:37:09.678844 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.678567    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19d2cd9d-54a4-44d9-b08c-45c3afa59146-tmp\") pod \"kube-auth-proxy-6ff87f57bc-md8k2\" (UID: \"19d2cd9d-54a4-44d9-b08c-45c3afa59146\") " pod="openshift-ingress/kube-auth-proxy-6ff87f57bc-md8k2"
Jul 30 17:37:09.678844 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.678589    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19d2cd9d-54a4-44d9-b08c-45c3afa59146-tls-certs\") pod \"kube-auth-proxy-6ff87f57bc-md8k2\" (UID: \"19d2cd9d-54a4-44d9-b08c-45c3afa59146\") " pod="openshift-ingress/kube-auth-proxy-6ff87f57bc-md8k2"
Jul 30 17:37:09.678844 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:37:09.678692    2577 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 30 17:37:09.678844 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:37:09.678759    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19d2cd9d-54a4-44d9-b08c-45c3afa59146-tls-certs podName:19d2cd9d-54a4-44d9-b08c-45c3afa59146 nodeName:}" failed. No retries permitted until 2026-07-30 17:37:10.1787341 +0000 UTC m=+604.632298071 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/19d2cd9d-54a4-44d9-b08c-45c3afa59146-tls-certs") pod "kube-auth-proxy-6ff87f57bc-md8k2" (UID: "19d2cd9d-54a4-44d9-b08c-45c3afa59146") : secret "kube-auth-proxy-tls" not found
Jul 30 17:37:09.681171 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.681148    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19d2cd9d-54a4-44d9-b08c-45c3afa59146-tmp\") pod \"kube-auth-proxy-6ff87f57bc-md8k2\" (UID: \"19d2cd9d-54a4-44d9-b08c-45c3afa59146\") " pod="openshift-ingress/kube-auth-proxy-6ff87f57bc-md8k2"
Jul 30 17:37:09.692646 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.692623    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phznb\" (UniqueName: \"kubernetes.io/projected/19d2cd9d-54a4-44d9-b08c-45c3afa59146-kube-api-access-phznb\") pod \"kube-auth-proxy-6ff87f57bc-md8k2\" (UID: \"19d2cd9d-54a4-44d9-b08c-45c3afa59146\") " pod="openshift-ingress/kube-auth-proxy-6ff87f57bc-md8k2"
Jul 30 17:37:09.738352 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.738317    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:09.779700 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.779668    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkjgx\" (UniqueName: \"kubernetes.io/projected/29a8ca15-2df6-4ba4-adfc-a0024f4c3648-kube-api-access-pkjgx\") pod \"odh-model-controller-7998c67dc4-jtrxh\" (UID: \"29a8ca15-2df6-4ba4-adfc-a0024f4c3648\") " pod="opendatahub/odh-model-controller-7998c67dc4-jtrxh"
Jul 30 17:37:09.779890 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.779755    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29a8ca15-2df6-4ba4-adfc-a0024f4c3648-cert\") pod \"odh-model-controller-7998c67dc4-jtrxh\" (UID: \"29a8ca15-2df6-4ba4-adfc-a0024f4c3648\") " pod="opendatahub/odh-model-controller-7998c67dc4-jtrxh"
Jul 30 17:37:09.779947 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:37:09.779897    2577 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 30 17:37:09.779995 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:37:09.779965    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29a8ca15-2df6-4ba4-adfc-a0024f4c3648-cert podName:29a8ca15-2df6-4ba4-adfc-a0024f4c3648 nodeName:}" failed. No retries permitted until 2026-07-30 17:37:10.279946972 +0000 UTC m=+604.733510968 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/29a8ca15-2df6-4ba4-adfc-a0024f4c3648-cert") pod "odh-model-controller-7998c67dc4-jtrxh" (UID: "29a8ca15-2df6-4ba4-adfc-a0024f4c3648") : secret "odh-model-controller-webhook-cert" not found
Jul 30 17:37:09.793437 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.793406    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkjgx\" (UniqueName: \"kubernetes.io/projected/29a8ca15-2df6-4ba4-adfc-a0024f4c3648-kube-api-access-pkjgx\") pod \"odh-model-controller-7998c67dc4-jtrxh\" (UID: \"29a8ca15-2df6-4ba4-adfc-a0024f4c3648\") " pod="opendatahub/odh-model-controller-7998c67dc4-jtrxh"
Jul 30 17:37:09.838273 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.838195    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:09.952283 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:09.952250    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:10.122348 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.122267    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:10.183797 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.183760    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19d2cd9d-54a4-44d9-b08c-45c3afa59146-tls-certs\") pod \"kube-auth-proxy-6ff87f57bc-md8k2\" (UID: \"19d2cd9d-54a4-44d9-b08c-45c3afa59146\") " pod="openshift-ingress/kube-auth-proxy-6ff87f57bc-md8k2"
Jul 30 17:37:10.186472 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.186446    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19d2cd9d-54a4-44d9-b08c-45c3afa59146-tls-certs\") pod \"kube-auth-proxy-6ff87f57bc-md8k2\" (UID: \"19d2cd9d-54a4-44d9-b08c-45c3afa59146\") " pod="openshift-ingress/kube-auth-proxy-6ff87f57bc-md8k2"
Jul 30 17:37:10.247224 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.247186    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:10.284721 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.284683    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29a8ca15-2df6-4ba4-adfc-a0024f4c3648-cert\") pod \"odh-model-controller-7998c67dc4-jtrxh\" (UID: \"29a8ca15-2df6-4ba4-adfc-a0024f4c3648\") " pod="opendatahub/odh-model-controller-7998c67dc4-jtrxh"
Jul 30 17:37:10.287263 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.287235    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29a8ca15-2df6-4ba4-adfc-a0024f4c3648-cert\") pod \"odh-model-controller-7998c67dc4-jtrxh\" (UID: \"29a8ca15-2df6-4ba4-adfc-a0024f4c3648\") " pod="opendatahub/odh-model-controller-7998c67dc4-jtrxh"
Jul 30 17:37:10.406743 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.406664    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6ff87f57bc-md8k2"
Jul 30 17:37:10.539461 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.539435    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-jtrxh"
Jul 30 17:37:10.546551 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.546360    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6ff87f57bc-md8k2"]
Jul 30 17:37:10.548657 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:37:10.548636    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19d2cd9d_54a4_44d9_b08c_45c3afa59146.slice/crio-d42872c5520d4eae9e8b05b8f8a18ccb3e0be487fb28c7a83955896b14b0474c WatchSource:0}: Error finding container d42872c5520d4eae9e8b05b8f8a18ccb3e0be487fb28c7a83955896b14b0474c: Status 404 returned error can't find the container with id d42872c5520d4eae9e8b05b8f8a18ccb3e0be487fb28c7a83955896b14b0474c
Jul 30 17:37:10.571456 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.571410    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:10.611723 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.611692    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vxzgh"]
Jul 30 17:37:10.615191 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.615165    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-vxzgh"
Jul 30 17:37:10.618324 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.618302    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 30 17:37:10.618501 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.618307    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-jr87n\""
Jul 30 17:37:10.632857 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.632809    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vxzgh"]
Jul 30 17:37:10.689076 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.688981    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ql8c\" (UniqueName: \"kubernetes.io/projected/e86cc99c-4b17-4534-845a-73c8d130dba9-kube-api-access-6ql8c\") pod \"model-serving-api-54d89bd5fc-vxzgh\" (UID: \"e86cc99c-4b17-4534-845a-73c8d130dba9\") " pod="opendatahub/model-serving-api-54d89bd5fc-vxzgh"
Jul 30 17:37:10.689414 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.689108    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e86cc99c-4b17-4534-845a-73c8d130dba9-tls-certs\") pod \"model-serving-api-54d89bd5fc-vxzgh\" (UID: \"e86cc99c-4b17-4534-845a-73c8d130dba9\") " pod="opendatahub/model-serving-api-54d89bd5fc-vxzgh"
Jul 30 17:37:10.696110 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.696084    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jtrxh"]
Jul 30 17:37:10.698953 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:37:10.698930    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29a8ca15_2df6_4ba4_adfc_a0024f4c3648.slice/crio-da453eecb5822ceb35744d736b143661951360fc8d48e742404a3d0ea5ab4fc7 WatchSource:0}: Error finding container da453eecb5822ceb35744d736b143661951360fc8d48e742404a3d0ea5ab4fc7: Status 404 returned error can't find the container with id da453eecb5822ceb35744d736b143661951360fc8d48e742404a3d0ea5ab4fc7
Jul 30 17:37:10.768635 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.768593    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:10.781833 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.781795    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"]
Jul 30 17:37:10.784340 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.784316    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:10.786579 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.786558    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 30 17:37:10.786709 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.786580    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 17:37:10.786770 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.786749    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 30 17:37:10.786854 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.786805    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 30 17:37:10.786854 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.786835    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-qxt7c\""
Jul 30 17:37:10.789598 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.789579    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ql8c\" (UniqueName: \"kubernetes.io/projected/e86cc99c-4b17-4534-845a-73c8d130dba9-kube-api-access-6ql8c\") pod \"model-serving-api-54d89bd5fc-vxzgh\" (UID: \"e86cc99c-4b17-4534-845a-73c8d130dba9\") " pod="opendatahub/model-serving-api-54d89bd5fc-vxzgh"
Jul 30 17:37:10.789715 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.789656    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e86cc99c-4b17-4534-845a-73c8d130dba9-tls-certs\") pod \"model-serving-api-54d89bd5fc-vxzgh\" (UID: \"e86cc99c-4b17-4534-845a-73c8d130dba9\") " pod="opendatahub/model-serving-api-54d89bd5fc-vxzgh"
Jul 30 17:37:10.793836 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.793793    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e86cc99c-4b17-4534-845a-73c8d130dba9-tls-certs\") pod \"model-serving-api-54d89bd5fc-vxzgh\" (UID: \"e86cc99c-4b17-4534-845a-73c8d130dba9\") " pod="opendatahub/model-serving-api-54d89bd5fc-vxzgh"
Jul 30 17:37:10.799452 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.799428    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"]
Jul 30 17:37:10.806196 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.806176    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ql8c\" (UniqueName: \"kubernetes.io/projected/e86cc99c-4b17-4534-845a-73c8d130dba9-kube-api-access-6ql8c\") pod \"model-serving-api-54d89bd5fc-vxzgh\" (UID: \"e86cc99c-4b17-4534-845a-73c8d130dba9\") " pod="opendatahub/model-serving-api-54d89bd5fc-vxzgh"
Jul 30 17:37:10.838946 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.838921    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:10.890854 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.890799    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-n42md\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:10.891029 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.890869    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-n42md\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:10.891029 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.890918    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-n42md\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:10.891029 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.890948    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-n42md\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:10.891029 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.891026    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-n42md\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:10.891193 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.891080    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-n42md\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:10.891193 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.891134    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rhsk\" (UniqueName: \"kubernetes.io/projected/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-kube-api-access-6rhsk\") pod \"istiod-openshift-gateway-7b7995d5cd-n42md\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:10.927672 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.927631    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-vxzgh"
Jul 30 17:37:10.992203 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.992171    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-n42md\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:10.992374 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.992244    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-n42md\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:10.992374 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.992311    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rhsk\" (UniqueName: \"kubernetes.io/projected/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-kube-api-access-6rhsk\") pod \"istiod-openshift-gateway-7b7995d5cd-n42md\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:10.992374 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.992353    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-n42md\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:10.992532 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.992379    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-n42md\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:10.992532 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.992411    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-n42md\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:10.992532 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.992439    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-n42md\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:10.993447 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.993383    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-n42md\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:10.995629 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.995568    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-n42md\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:10.995751 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.995632    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-n42md\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:10.995848 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.995774    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-n42md\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:10.996392 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:10.996366    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-n42md\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:11.013131 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:11.013096    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-n42md\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:11.014834 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:11.014793    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rhsk\" (UniqueName: \"kubernetes.io/projected/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-kube-api-access-6rhsk\") pod \"istiod-openshift-gateway-7b7995d5cd-n42md\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:11.079442 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:11.079407    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vxzgh"]
Jul 30 17:37:11.082639 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:37:11.082607    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode86cc99c_4b17_4534_845a_73c8d130dba9.slice/crio-1ada603b62b8725383a3de1e2abdb30075c3dd9615166c9c973dced991cf1de5 WatchSource:0}: Error finding container 1ada603b62b8725383a3de1e2abdb30075c3dd9615166c9c973dced991cf1de5: Status 404 returned error can't find the container with id 1ada603b62b8725383a3de1e2abdb30075c3dd9615166c9c973dced991cf1de5
Jul 30 17:37:11.104790 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:11.104764    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:11.266345 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:11.266313    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"]
Jul 30 17:37:11.269868 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:37:11.269800    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfc8f5b9_4be1_4f2b_8dbb_264a1b516cc6.slice/crio-1c4c5464f7fe72b72d07794c6c1710955dff7dacd85b4a2aa027f3fd6203a07b WatchSource:0}: Error finding container 1c4c5464f7fe72b72d07794c6c1710955dff7dacd85b4a2aa027f3fd6203a07b: Status 404 returned error can't find the container with id 1c4c5464f7fe72b72d07794c6c1710955dff7dacd85b4a2aa027f3fd6203a07b
Jul 30 17:37:11.382541 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:11.382501    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-vxzgh" event={"ID":"e86cc99c-4b17-4534-845a-73c8d130dba9","Type":"ContainerStarted","Data":"1ada603b62b8725383a3de1e2abdb30075c3dd9615166c9c973dced991cf1de5"}
Jul 30 17:37:11.383860 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:11.383832    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-jtrxh" event={"ID":"29a8ca15-2df6-4ba4-adfc-a0024f4c3648","Type":"ContainerStarted","Data":"da453eecb5822ceb35744d736b143661951360fc8d48e742404a3d0ea5ab4fc7"}
Jul 30 17:37:11.385364 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:11.385302    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6ff87f57bc-md8k2" event={"ID":"19d2cd9d-54a4-44d9-b08c-45c3afa59146","Type":"ContainerStarted","Data":"d42872c5520d4eae9e8b05b8f8a18ccb3e0be487fb28c7a83955896b14b0474c"}
Jul 30 17:37:11.388414 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:11.388389    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md" event={"ID":"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6","Type":"ContainerStarted","Data":"1c4c5464f7fe72b72d07794c6c1710955dff7dacd85b4a2aa027f3fd6203a07b"}
Jul 30 17:37:11.855053 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:11.854975    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:15.413433 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:15.413069    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-vxzgh" event={"ID":"e86cc99c-4b17-4534-845a-73c8d130dba9","Type":"ContainerStarted","Data":"be7e1e764ccfedce30532b61a843a474229239dcb49c1b743b91034f2f05bb87"}
Jul 30 17:37:15.413433 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:15.413182    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-vxzgh"
Jul 30 17:37:15.415046 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:15.415015    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-jtrxh" event={"ID":"29a8ca15-2df6-4ba4-adfc-a0024f4c3648","Type":"ContainerStarted","Data":"4e5fdf31c8c45916ce256083c9c160315142de184cdb82efdb2cf90401b61f9d"}
Jul 30 17:37:15.415189 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:15.415150    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-jtrxh"
Jul 30 17:37:15.417033 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:15.416840    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6ff87f57bc-md8k2" event={"ID":"19d2cd9d-54a4-44d9-b08c-45c3afa59146","Type":"ContainerStarted","Data":"39d0063a1b1dbc06591a44378a1d93a15e442f219dc7bb627aaaec685de9a392"}
Jul 30 17:37:15.432584 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:15.432193    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-vxzgh" podStartSLOduration=1.7489921480000001 podStartE2EDuration="5.432172489s" totalImagesPullingTime="3.683180341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:37:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:37:15.431303782 +0000 UTC m=+609.884867774" watchObservedRunningTime="2026-07-30 17:37:15.432172489 +0000 UTC m=+609.885736487"
Jul 30 17:37:15.448317 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:15.448266    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6ff87f57bc-md8k2" podStartSLOduration=2.23570761 podStartE2EDuration="6.448246147s" totalImagesPullingTime="4.212538537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:37:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:37:15.446540393 +0000 UTC m=+609.900104388" watchObservedRunningTime="2026-07-30 17:37:15.448246147 +0000 UTC m=+609.901810139"
Jul 30 17:37:15.464373 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:15.464317    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-jtrxh" podStartSLOduration=2.400716214 podStartE2EDuration="6.464297497s" totalImagesPullingTime="4.063581283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:37:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:37:15.462916309 +0000 UTC m=+609.916480301" watchObservedRunningTime="2026-07-30 17:37:15.464297497 +0000 UTC m=+609.917861489"
Jul 30 17:37:16.155085 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:16.155045    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 30 17:37:16.155208 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:16.155137    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 30 17:37:16.424294 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:16.424163    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md" event={"ID":"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6","Type":"ContainerStarted","Data":"3bb1d67a0daabfe610bac6054a9f21ba6fb08f5ee7febccfdfec9f65e8484973"}
Jul 30 17:37:16.449930 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:16.449867    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md" podStartSLOduration=1.567425887 podStartE2EDuration="6.449808027s" totalImagesPullingTime="4.88238214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:37:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:37:16.447442323 +0000 UTC m=+610.901006326" watchObservedRunningTime="2026-07-30 17:37:16.449808027 +0000 UTC m=+610.903372019"
Jul 30 17:37:16.595949 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:16.595915    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:16.698227 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:16.698137    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:17.054378 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:17.054349    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:17.297417 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:17.297380    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:17.399152 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:17.399057    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:17.431480 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:17.431453    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:17.433246 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:17.433223    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:21.134646 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.134616    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:21.238234 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.238194    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mchct"]
Jul 30 17:37:21.241725 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.241706    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-98m6c"]
Jul 30 17:37:21.241895 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.241872    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mchct"
Jul 30 17:37:21.244449 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.244424    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 30 17:37:21.244652 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.244630    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-mxflm\""
Jul 30 17:37:21.245003 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.244983    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-98m6c"
Jul 30 17:37:21.247572 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.247547    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 30 17:37:21.247660 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.247617    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-8r4p6\""
Jul 30 17:37:21.258924 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.258900    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-98m6c"]
Jul 30 17:37:21.260674 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.260655    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mchct"]
Jul 30 17:37:21.282460 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.282429    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:21.297428 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.297401    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjgh9\" (UniqueName: \"kubernetes.io/projected/b1596812-1f98-42ab-ace0-01fd85f11a4c-kube-api-access-kjgh9\") pod \"kserve-controller-manager-d6f4b65b8-mchct\" (UID: \"b1596812-1f98-42ab-ace0-01fd85f11a4c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mchct"
Jul 30 17:37:21.297566 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.297441    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1596812-1f98-42ab-ace0-01fd85f11a4c-cert\") pod \"kserve-controller-manager-d6f4b65b8-mchct\" (UID: \"b1596812-1f98-42ab-ace0-01fd85f11a4c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mchct"
Jul 30 17:37:21.297566 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.297472    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/421c5189-15a2-4509-9c61-2ffbfdd35204-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-98m6c\" (UID: \"421c5189-15a2-4509-9c61-2ffbfdd35204\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-98m6c"
Jul 30 17:37:21.297566 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.297506    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hrd6\" (UniqueName: \"kubernetes.io/projected/421c5189-15a2-4509-9c61-2ffbfdd35204-kube-api-access-8hrd6\") pod \"llmisvc-controller-manager-777c6bd7d5-98m6c\" (UID: \"421c5189-15a2-4509-9c61-2ffbfdd35204\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-98m6c"
Jul 30 17:37:21.398632 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.398556    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjgh9\" (UniqueName: \"kubernetes.io/projected/b1596812-1f98-42ab-ace0-01fd85f11a4c-kube-api-access-kjgh9\") pod \"kserve-controller-manager-d6f4b65b8-mchct\" (UID: \"b1596812-1f98-42ab-ace0-01fd85f11a4c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mchct"
Jul 30 17:37:21.398632 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.398599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1596812-1f98-42ab-ace0-01fd85f11a4c-cert\") pod \"kserve-controller-manager-d6f4b65b8-mchct\" (UID: \"b1596812-1f98-42ab-ace0-01fd85f11a4c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mchct"
Jul 30 17:37:21.398912 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.398646    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/421c5189-15a2-4509-9c61-2ffbfdd35204-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-98m6c\" (UID: \"421c5189-15a2-4509-9c61-2ffbfdd35204\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-98m6c"
Jul 30 17:37:21.398912 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.398676    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hrd6\" (UniqueName: \"kubernetes.io/projected/421c5189-15a2-4509-9c61-2ffbfdd35204-kube-api-access-8hrd6\") pod \"llmisvc-controller-manager-777c6bd7d5-98m6c\" (UID: \"421c5189-15a2-4509-9c61-2ffbfdd35204\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-98m6c"
Jul 30 17:37:21.398912 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:37:21.398783    2577 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 30 17:37:21.398912 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:37:21.398801    2577 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 30 17:37:21.398912 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:37:21.398899    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1596812-1f98-42ab-ace0-01fd85f11a4c-cert podName:b1596812-1f98-42ab-ace0-01fd85f11a4c nodeName:}" failed. No retries permitted until 2026-07-30 17:37:21.898877412 +0000 UTC m=+616.352441385 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b1596812-1f98-42ab-ace0-01fd85f11a4c-cert") pod "kserve-controller-manager-d6f4b65b8-mchct" (UID: "b1596812-1f98-42ab-ace0-01fd85f11a4c") : secret "kserve-webhook-server-cert" not found
Jul 30 17:37:21.398912 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:37:21.398916    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/421c5189-15a2-4509-9c61-2ffbfdd35204-cert podName:421c5189-15a2-4509-9c61-2ffbfdd35204 nodeName:}" failed. No retries permitted until 2026-07-30 17:37:21.898909378 +0000 UTC m=+616.352473347 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/421c5189-15a2-4509-9c61-2ffbfdd35204-cert") pod "llmisvc-controller-manager-777c6bd7d5-98m6c" (UID: "421c5189-15a2-4509-9c61-2ffbfdd35204") : secret "llmisvc-webhook-server-cert" not found
Jul 30 17:37:21.410310 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.410280    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hrd6\" (UniqueName: \"kubernetes.io/projected/421c5189-15a2-4509-9c61-2ffbfdd35204-kube-api-access-8hrd6\") pod \"llmisvc-controller-manager-777c6bd7d5-98m6c\" (UID: \"421c5189-15a2-4509-9c61-2ffbfdd35204\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-98m6c"
Jul 30 17:37:21.411955 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.411936    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjgh9\" (UniqueName: \"kubernetes.io/projected/b1596812-1f98-42ab-ace0-01fd85f11a4c-kube-api-access-kjgh9\") pod \"kserve-controller-manager-d6f4b65b8-mchct\" (UID: \"b1596812-1f98-42ab-ace0-01fd85f11a4c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mchct"
Jul 30 17:37:21.734168 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.734078    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:21.853902 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.853873    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:21.902684 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.902660    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1596812-1f98-42ab-ace0-01fd85f11a4c-cert\") pod \"kserve-controller-manager-d6f4b65b8-mchct\" (UID: \"b1596812-1f98-42ab-ace0-01fd85f11a4c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mchct"
Jul 30 17:37:21.902808 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.902699    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/421c5189-15a2-4509-9c61-2ffbfdd35204-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-98m6c\" (UID: \"421c5189-15a2-4509-9c61-2ffbfdd35204\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-98m6c"
Jul 30 17:37:21.905249 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.905226    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1596812-1f98-42ab-ace0-01fd85f11a4c-cert\") pod \"kserve-controller-manager-d6f4b65b8-mchct\" (UID: \"b1596812-1f98-42ab-ace0-01fd85f11a4c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mchct"
Jul 30 17:37:21.905341 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:21.905278    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/421c5189-15a2-4509-9c61-2ffbfdd35204-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-98m6c\" (UID: \"421c5189-15a2-4509-9c61-2ffbfdd35204\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-98m6c"
Jul 30 17:37:22.153987 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:22.153954    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mchct"
Jul 30 17:37:22.161009 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:22.160992    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-98m6c"
Jul 30 17:37:22.303900 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:22.303878    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mchct"]
Jul 30 17:37:22.305811 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:37:22.305786    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1596812_1f98_42ab_ace0_01fd85f11a4c.slice/crio-8e3a89f7f629d04efd79bd22c240800cda6810100628ee6e3f36bfe6eafc4b9e WatchSource:0}: Error finding container 8e3a89f7f629d04efd79bd22c240800cda6810100628ee6e3f36bfe6eafc4b9e: Status 404 returned error can't find the container with id 8e3a89f7f629d04efd79bd22c240800cda6810100628ee6e3f36bfe6eafc4b9e
Jul 30 17:37:22.328564 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:22.328507    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-98m6c"]
Jul 30 17:37:22.330622 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:37:22.330590    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod421c5189_15a2_4509_9c61_2ffbfdd35204.slice/crio-361dc7ab30869f80b3185beb74476197c4b0550d916d0d90511f4c6190e47428 WatchSource:0}: Error finding container 361dc7ab30869f80b3185beb74476197c4b0550d916d0d90511f4c6190e47428: Status 404 returned error can't find the container with id 361dc7ab30869f80b3185beb74476197c4b0550d916d0d90511f4c6190e47428
Jul 30 17:37:22.456221 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:22.456134    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-98m6c" event={"ID":"421c5189-15a2-4509-9c61-2ffbfdd35204","Type":"ContainerStarted","Data":"361dc7ab30869f80b3185beb74476197c4b0550d916d0d90511f4c6190e47428"}
Jul 30 17:37:22.456956 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:22.456933    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:22.457355 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:22.457335    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mchct" event={"ID":"b1596812-1f98-42ab-ace0-01fd85f11a4c","Type":"ContainerStarted","Data":"8e3a89f7f629d04efd79bd22c240800cda6810100628ee6e3f36bfe6eafc4b9e"}
Jul 30 17:37:22.556041 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:22.556002    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:22.653215 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:22.653183    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:23.289273 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:23.289237    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:23.349317 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:23.349288    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:37:24.130923 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:24.130890    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"]
Jul 30 17:37:24.131150 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:24.131116    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md" podUID="bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6" containerName="discovery" containerID="cri-o://3bb1d67a0daabfe610bac6054a9f21ba6fb08f5ee7febccfdfec9f65e8484973" gracePeriod=30
Jul 30 17:37:24.473262 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:24.473031    2577 generic.go:356] "Generic (PLEG): container finished" podID="bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6" containerID="3bb1d67a0daabfe610bac6054a9f21ba6fb08f5ee7febccfdfec9f65e8484973" exitCode=0
Jul 30 17:37:24.473262 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:24.473164    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md" event={"ID":"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6","Type":"ContainerDied","Data":"3bb1d67a0daabfe610bac6054a9f21ba6fb08f5ee7febccfdfec9f65e8484973"}
Jul 30 17:37:25.082078 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.082052    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:25.238098 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.238065    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-cacerts\") pod \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") "
Jul 30 17:37:25.238258 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.238108    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-local-certs\") pod \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") "
Jul 30 17:37:25.238258 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.238125    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-token\") pod \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") "
Jul 30 17:37:25.238258 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.238156    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-csr-dns-cert\") pod \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") "
Jul 30 17:37:25.238258 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.238206    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rhsk\" (UniqueName: \"kubernetes.io/projected/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-kube-api-access-6rhsk\") pod \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") "
Jul 30 17:37:25.238258 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.238251    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-csr-ca-configmap\") pod \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") "
Jul 30 17:37:25.238490 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.238398    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-kubeconfig\") pod \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\" (UID: \"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6\") "
Jul 30 17:37:25.238650 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.238612    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6" (UID: "bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:37:25.238725 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.238708    2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-csr-ca-configmap\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:37:25.240830 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.240776    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6" (UID: "bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:37:25.240954 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.240898    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-cacerts" (OuterVolumeSpecName: "cacerts") pod "bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6" (UID: "bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:37:25.240954 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.240933    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-kube-api-access-6rhsk" (OuterVolumeSpecName: "kube-api-access-6rhsk") pod "bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6" (UID: "bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6"). InnerVolumeSpecName "kube-api-access-6rhsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:37:25.241043 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.240942    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-token" (OuterVolumeSpecName: "istio-token") pod "bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6" (UID: "bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:37:25.241139 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.241124    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-local-certs" (OuterVolumeSpecName: "local-certs") pod "bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6" (UID: "bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:37:25.241177 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.241138    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6" (UID: "bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:37:25.339985 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.339894    2577 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-kubeconfig\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:37:25.339985 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.339928    2577 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-cacerts\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:37:25.339985 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.339938    2577 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-local-certs\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:37:25.339985 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.339946    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-token\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:37:25.339985 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.339955    2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-istio-csr-dns-cert\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:37:25.339985 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.339963    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rhsk\" (UniqueName: \"kubernetes.io/projected/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6-kube-api-access-6rhsk\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:37:25.479605 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.479563    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mchct" event={"ID":"b1596812-1f98-42ab-ace0-01fd85f11a4c","Type":"ContainerStarted","Data":"40014318bfd7602baa0bf5006e6b81be0a0dac82eeebf5c63600f5af34fe86b1"}
Jul 30 17:37:25.480081 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.479663    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mchct"
Jul 30 17:37:25.480966 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.480949    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"
Jul 30 17:37:25.481096 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.480976    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md" event={"ID":"bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6","Type":"ContainerDied","Data":"1c4c5464f7fe72b72d07794c6c1710955dff7dacd85b4a2aa027f3fd6203a07b"}
Jul 30 17:37:25.481096 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.481028    2577 scope.go:117] "RemoveContainer" containerID="3bb1d67a0daabfe610bac6054a9f21ba6fb08f5ee7febccfdfec9f65e8484973"
Jul 30 17:37:25.482628 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.482603    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-98m6c" event={"ID":"421c5189-15a2-4509-9c61-2ffbfdd35204","Type":"ContainerStarted","Data":"7500de15a6ac94c37bd28712cc6a2aec88f511622549067c8d4d8d88cc86f842"}
Jul 30 17:37:25.482798 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.482758    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-98m6c"
Jul 30 17:37:25.500199 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.500146    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mchct" podStartSLOduration=2.360944689 podStartE2EDuration="4.500130572s" totalImagesPullingTime="2.139185883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:37:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:37:25.498525209 +0000 UTC m=+619.952089202" watchObservedRunningTime="2026-07-30 17:37:25.500130572 +0000 UTC m=+619.953694567"
Jul 30 17:37:25.521381 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.521337    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-98m6c" podStartSLOduration=1.741893624 podStartE2EDuration="4.52132188s" totalImagesPullingTime="2.779428256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:37:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:37:25.519113808 +0000 UTC m=+619.972677799" watchObservedRunningTime="2026-07-30 17:37:25.52132188 +0000 UTC m=+619.974885870"
Jul 30 17:37:25.537753 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.537727    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"]
Jul 30 17:37:25.543561 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:25.543535    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-n42md"]
Jul 30 17:37:26.110134 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:26.110087    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6" path="/var/lib/kubelet/pods/bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6/volumes"
Jul 30 17:37:26.426410 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:26.426332    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-jtrxh"
Jul 30 17:37:26.428518 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:26.428498    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-vxzgh"
Jul 30 17:37:56.492000 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:56.491970    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-98m6c"
Jul 30 17:37:56.495162 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:37:56.495127    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mchct"
Jul 30 17:38:10.265838 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:10.265775    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:10.366539 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:10.366510    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:10.403431 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:10.403398    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xx2vr"]
Jul 30 17:38:10.403742 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:10.403730    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6" containerName="discovery"
Jul 30 17:38:10.403784 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:10.403743    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6" containerName="discovery"
Jul 30 17:38:10.403833 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:10.403803    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfc8f5b9-4be1-4f2b-8dbb-264a1b516cc6" containerName="discovery"
Jul 30 17:38:10.406861 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:10.406838    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xx2vr"
Jul 30 17:38:10.409029 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:10.409008    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 30 17:38:10.409546 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:10.409526    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 30 17:38:10.409656 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:10.409529    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-45gcg\""
Jul 30 17:38:10.417667 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:10.417638    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xx2vr"]
Jul 30 17:38:10.460209 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:10.460181    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:10.519514 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:10.519440    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgmws\" (UniqueName: \"kubernetes.io/projected/dbef5c2e-0b90-4f5c-9570-fadb3c6eb976-kube-api-access-zgmws\") pod \"kuadrant-operator-catalog-xx2vr\" (UID: \"dbef5c2e-0b90-4f5c-9570-fadb3c6eb976\") " pod="kuadrant-system/kuadrant-operator-catalog-xx2vr"
Jul 30 17:38:10.620605 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:10.620573    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgmws\" (UniqueName: \"kubernetes.io/projected/dbef5c2e-0b90-4f5c-9570-fadb3c6eb976-kube-api-access-zgmws\") pod \"kuadrant-operator-catalog-xx2vr\" (UID: \"dbef5c2e-0b90-4f5c-9570-fadb3c6eb976\") " pod="kuadrant-system/kuadrant-operator-catalog-xx2vr"
Jul 30 17:38:10.628993 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:10.628970    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgmws\" (UniqueName: \"kubernetes.io/projected/dbef5c2e-0b90-4f5c-9570-fadb3c6eb976-kube-api-access-zgmws\") pod \"kuadrant-operator-catalog-xx2vr\" (UID: \"dbef5c2e-0b90-4f5c-9570-fadb3c6eb976\") " pod="kuadrant-system/kuadrant-operator-catalog-xx2vr"
Jul 30 17:38:10.718411 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:10.718377    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xx2vr"
Jul 30 17:38:10.758796 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:10.758755    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xx2vr"]
Jul 30 17:38:10.851678 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:10.851625    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xx2vr"]
Jul 30 17:38:10.855562 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:38:10.855534    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbef5c2e_0b90_4f5c_9570_fadb3c6eb976.slice/crio-c066d90748ed65d86997f1bf6c00cc0fe90ae0b2b29d6ec9aacc56fe4cc00202 WatchSource:0}: Error finding container c066d90748ed65d86997f1bf6c00cc0fe90ae0b2b29d6ec9aacc56fe4cc00202: Status 404 returned error can't find the container with id c066d90748ed65d86997f1bf6c00cc0fe90ae0b2b29d6ec9aacc56fe4cc00202
Jul 30 17:38:10.856948 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:10.856932    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 17:38:10.971241 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:10.971204    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-95lpf"]
Jul 30 17:38:10.974741 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:10.974722    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-95lpf"
Jul 30 17:38:10.983535 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:10.983502    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-95lpf"]
Jul 30 17:38:11.124738 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:11.124657    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m625\" (UniqueName: \"kubernetes.io/projected/5237a970-40b8-4b10-b4a1-237e27d0a2af-kube-api-access-9m625\") pod \"kuadrant-operator-catalog-95lpf\" (UID: \"5237a970-40b8-4b10-b4a1-237e27d0a2af\") " pod="kuadrant-system/kuadrant-operator-catalog-95lpf"
Jul 30 17:38:11.218192 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:11.218161    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:11.225325 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:11.225296    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m625\" (UniqueName: \"kubernetes.io/projected/5237a970-40b8-4b10-b4a1-237e27d0a2af-kube-api-access-9m625\") pod \"kuadrant-operator-catalog-95lpf\" (UID: \"5237a970-40b8-4b10-b4a1-237e27d0a2af\") " pod="kuadrant-system/kuadrant-operator-catalog-95lpf"
Jul 30 17:38:11.233858 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:11.233810    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m625\" (UniqueName: \"kubernetes.io/projected/5237a970-40b8-4b10-b4a1-237e27d0a2af-kube-api-access-9m625\") pod \"kuadrant-operator-catalog-95lpf\" (UID: \"5237a970-40b8-4b10-b4a1-237e27d0a2af\") " pod="kuadrant-system/kuadrant-operator-catalog-95lpf"
Jul 30 17:38:11.284227 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:11.284205    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-95lpf"
Jul 30 17:38:11.401541 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:11.401470    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:11.430139 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:11.430113    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-95lpf"]
Jul 30 17:38:11.470773 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:38:11.470743    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5237a970_40b8_4b10_b4a1_237e27d0a2af.slice/crio-d7e9c99561a64a7a776102a9854965ddf3a19cb2ec8d3212e43324fe1b012543 WatchSource:0}: Error finding container d7e9c99561a64a7a776102a9854965ddf3a19cb2ec8d3212e43324fe1b012543: Status 404 returned error can't find the container with id d7e9c99561a64a7a776102a9854965ddf3a19cb2ec8d3212e43324fe1b012543
Jul 30 17:38:11.695197 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:11.695103    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-95lpf" event={"ID":"5237a970-40b8-4b10-b4a1-237e27d0a2af","Type":"ContainerStarted","Data":"d7e9c99561a64a7a776102a9854965ddf3a19cb2ec8d3212e43324fe1b012543"}
Jul 30 17:38:11.696566 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:11.696537    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xx2vr" event={"ID":"dbef5c2e-0b90-4f5c-9570-fadb3c6eb976","Type":"ContainerStarted","Data":"c066d90748ed65d86997f1bf6c00cc0fe90ae0b2b29d6ec9aacc56fe4cc00202"}
Jul 30 17:38:13.708063 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:13.708018    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xx2vr" event={"ID":"dbef5c2e-0b90-4f5c-9570-fadb3c6eb976","Type":"ContainerStarted","Data":"d4fb0cfbd7dc6b2539a060de2fde8e8ca29bc1c93ae4ddbccade0ce5ef0340f8"}
Jul 30 17:38:13.708063 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:13.708049    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-xx2vr" podUID="dbef5c2e-0b90-4f5c-9570-fadb3c6eb976" containerName="registry-server" containerID="cri-o://d4fb0cfbd7dc6b2539a060de2fde8e8ca29bc1c93ae4ddbccade0ce5ef0340f8" gracePeriod=2
Jul 30 17:38:13.709537 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:13.709514    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-95lpf" event={"ID":"5237a970-40b8-4b10-b4a1-237e27d0a2af","Type":"ContainerStarted","Data":"af2cc3cf41edf035c32c185d7e60e9ef156e3639b2007932f06bf70462743a71"}
Jul 30 17:38:13.715471 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:13.715452    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:13.724166 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:13.724126    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-xx2vr" podStartSLOduration=1.787681784 podStartE2EDuration="3.724111243s" totalImagesPullingTime="1.936429459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:38:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:38:13.722542675 +0000 UTC m=+668.176106667" watchObservedRunningTime="2026-07-30 17:38:13.724111243 +0000 UTC m=+668.177675234"
Jul 30 17:38:13.739906 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:13.739865    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-95lpf" podStartSLOduration=2.417926608 podStartE2EDuration="3.739849535s" totalImagesPullingTime="1.321922927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:38:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:38:13.738034886 +0000 UTC m=+668.191598877" watchObservedRunningTime="2026-07-30 17:38:13.739849535 +0000 UTC m=+668.193413530"
Jul 30 17:38:13.815860 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:13.815812    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:13.956031 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:13.956007    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xx2vr"
Jul 30 17:38:14.049628 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:14.049585    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgmws\" (UniqueName: \"kubernetes.io/projected/dbef5c2e-0b90-4f5c-9570-fadb3c6eb976-kube-api-access-zgmws\") pod \"dbef5c2e-0b90-4f5c-9570-fadb3c6eb976\" (UID: \"dbef5c2e-0b90-4f5c-9570-fadb3c6eb976\") "
Jul 30 17:38:14.051900 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:14.051875    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbef5c2e-0b90-4f5c-9570-fadb3c6eb976-kube-api-access-zgmws" (OuterVolumeSpecName: "kube-api-access-zgmws") pod "dbef5c2e-0b90-4f5c-9570-fadb3c6eb976" (UID: "dbef5c2e-0b90-4f5c-9570-fadb3c6eb976"). InnerVolumeSpecName "kube-api-access-zgmws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:38:14.150893 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:14.150864    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgmws\" (UniqueName: \"kubernetes.io/projected/dbef5c2e-0b90-4f5c-9570-fadb3c6eb976-kube-api-access-zgmws\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:38:14.714703 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:14.714668    2577 generic.go:356] "Generic (PLEG): container finished" podID="dbef5c2e-0b90-4f5c-9570-fadb3c6eb976" containerID="d4fb0cfbd7dc6b2539a060de2fde8e8ca29bc1c93ae4ddbccade0ce5ef0340f8" exitCode=0
Jul 30 17:38:14.715161 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:14.714713    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xx2vr"
Jul 30 17:38:14.715161 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:14.714753    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xx2vr" event={"ID":"dbef5c2e-0b90-4f5c-9570-fadb3c6eb976","Type":"ContainerDied","Data":"d4fb0cfbd7dc6b2539a060de2fde8e8ca29bc1c93ae4ddbccade0ce5ef0340f8"}
Jul 30 17:38:14.715161 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:14.714800    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xx2vr" event={"ID":"dbef5c2e-0b90-4f5c-9570-fadb3c6eb976","Type":"ContainerDied","Data":"c066d90748ed65d86997f1bf6c00cc0fe90ae0b2b29d6ec9aacc56fe4cc00202"}
Jul 30 17:38:14.715161 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:14.714831    2577 scope.go:117] "RemoveContainer" containerID="d4fb0cfbd7dc6b2539a060de2fde8e8ca29bc1c93ae4ddbccade0ce5ef0340f8"
Jul 30 17:38:14.723441 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:14.723426    2577 scope.go:117] "RemoveContainer" containerID="d4fb0cfbd7dc6b2539a060de2fde8e8ca29bc1c93ae4ddbccade0ce5ef0340f8"
Jul 30 17:38:14.723700 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:38:14.723681    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4fb0cfbd7dc6b2539a060de2fde8e8ca29bc1c93ae4ddbccade0ce5ef0340f8\": container with ID starting with d4fb0cfbd7dc6b2539a060de2fde8e8ca29bc1c93ae4ddbccade0ce5ef0340f8 not found: ID does not exist" containerID="d4fb0cfbd7dc6b2539a060de2fde8e8ca29bc1c93ae4ddbccade0ce5ef0340f8"
Jul 30 17:38:14.723762 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:14.723712    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4fb0cfbd7dc6b2539a060de2fde8e8ca29bc1c93ae4ddbccade0ce5ef0340f8"} err="failed to get container status \"d4fb0cfbd7dc6b2539a060de2fde8e8ca29bc1c93ae4ddbccade0ce5ef0340f8\": rpc error: code = NotFound desc = could not find container \"d4fb0cfbd7dc6b2539a060de2fde8e8ca29bc1c93ae4ddbccade0ce5ef0340f8\": container with ID starting with d4fb0cfbd7dc6b2539a060de2fde8e8ca29bc1c93ae4ddbccade0ce5ef0340f8 not found: ID does not exist"
Jul 30 17:38:14.730297 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:14.730274    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xx2vr"]
Jul 30 17:38:14.736052 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:14.736031    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xx2vr"]
Jul 30 17:38:16.109479 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:16.109434    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbef5c2e-0b90-4f5c-9570-fadb3c6eb976" path="/var/lib/kubelet/pods/dbef5c2e-0b90-4f5c-9570-fadb3c6eb976/volumes"
Jul 30 17:38:21.284682 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:21.284650    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-95lpf"
Jul 30 17:38:21.285134 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:21.284702    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-95lpf"
Jul 30 17:38:21.306636 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:21.306611    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-95lpf"
Jul 30 17:38:21.765211 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:21.765187    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-95lpf"
Jul 30 17:38:33.984085 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:33.984038    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:37.381902 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:37.381862    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:38.707610 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.707576    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75449b9c89-dwplx"]
Jul 30 17:38:38.708048 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.707961    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbef5c2e-0b90-4f5c-9570-fadb3c6eb976" containerName="registry-server"
Jul 30 17:38:38.708048 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.707975    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbef5c2e-0b90-4f5c-9570-fadb3c6eb976" containerName="registry-server"
Jul 30 17:38:38.708048 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.708024    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbef5c2e-0b90-4f5c-9570-fadb3c6eb976" containerName="registry-server"
Jul 30 17:38:38.713929 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.713913    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:38.717778 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.717755    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 30 17:38:38.718539 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.718525    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 30 17:38:38.718619 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.718603    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7zw78\""
Jul 30 17:38:38.719456 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.719440    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 30 17:38:38.719731 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.719695    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 30 17:38:38.719865 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.719751    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 30 17:38:38.719865 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.719746    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 30 17:38:38.722938 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.722923    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 30 17:38:38.728160 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.728141    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm"]
Jul 30 17:38:38.729279 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.729257    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 30 17:38:38.731410 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.731395    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm"
Jul 30 17:38:38.734684 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.734667    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-lw975\""
Jul 30 17:38:38.746448 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.746425    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75449b9c89-dwplx"]
Jul 30 17:38:38.758961 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.758935    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbz4m\" (UniqueName: \"kubernetes.io/projected/71d7f7c4-a64e-4aa9-95bf-aa89365595c6-kube-api-access-vbz4m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-42rjm\" (UID: \"71d7f7c4-a64e-4aa9-95bf-aa89365595c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm"
Jul 30 17:38:38.759110 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.759030    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77-console-serving-cert\") pod \"console-75449b9c89-dwplx\" (UID: \"17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77\") " pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:38.759110 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.759061    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77-service-ca\") pod \"console-75449b9c89-dwplx\" (UID: \"17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77\") " pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:38.759110 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.759087    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77-oauth-serving-cert\") pod \"console-75449b9c89-dwplx\" (UID: \"17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77\") " pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:38.759241 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.759166    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77-trusted-ca-bundle\") pod \"console-75449b9c89-dwplx\" (UID: \"17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77\") " pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:38.759241 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.759202    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77-console-oauth-config\") pod \"console-75449b9c89-dwplx\" (UID: \"17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77\") " pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:38.759241 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.759223    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71d7f7c4-a64e-4aa9-95bf-aa89365595c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-42rjm\" (UID: \"71d7f7c4-a64e-4aa9-95bf-aa89365595c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm"
Jul 30 17:38:38.759241 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.759240    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lq4v\" (UniqueName: \"kubernetes.io/projected/17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77-kube-api-access-7lq4v\") pod \"console-75449b9c89-dwplx\" (UID: \"17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77\") " pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:38.759399 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.759266    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77-console-config\") pod \"console-75449b9c89-dwplx\" (UID: \"17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77\") " pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:38.770898 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.770859    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm"]
Jul 30 17:38:38.856863 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.856809    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:38.860318 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.860300    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77-console-serving-cert\") pod \"console-75449b9c89-dwplx\" (UID: \"17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77\") " pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:38.860391 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.860328    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77-service-ca\") pod \"console-75449b9c89-dwplx\" (UID: \"17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77\") " pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:38.860391 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.860350    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77-oauth-serving-cert\") pod \"console-75449b9c89-dwplx\" (UID: \"17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77\") " pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:38.860490 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.860473    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77-trusted-ca-bundle\") pod \"console-75449b9c89-dwplx\" (UID: \"17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77\") " pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:38.860526 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.860508    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77-console-oauth-config\") pod \"console-75449b9c89-dwplx\" (UID: \"17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77\") " pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:38.860569 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.860529    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71d7f7c4-a64e-4aa9-95bf-aa89365595c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-42rjm\" (UID: \"71d7f7c4-a64e-4aa9-95bf-aa89365595c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm"
Jul 30 17:38:38.860569 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.860561    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lq4v\" (UniqueName: \"kubernetes.io/projected/17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77-kube-api-access-7lq4v\") pod \"console-75449b9c89-dwplx\" (UID: \"17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77\") " pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:38.860789 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.860759    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77-console-config\") pod \"console-75449b9c89-dwplx\" (UID: \"17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77\") " pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:38.860907 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.860835    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbz4m\" (UniqueName: \"kubernetes.io/projected/71d7f7c4-a64e-4aa9-95bf-aa89365595c6-kube-api-access-vbz4m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-42rjm\" (UID: \"71d7f7c4-a64e-4aa9-95bf-aa89365595c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm"
Jul 30 17:38:38.861146 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.861119    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71d7f7c4-a64e-4aa9-95bf-aa89365595c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-42rjm\" (UID: \"71d7f7c4-a64e-4aa9-95bf-aa89365595c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm"
Jul 30 17:38:38.861309 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.861152    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77-service-ca\") pod \"console-75449b9c89-dwplx\" (UID: \"17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77\") " pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:38.861309 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.861208    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77-oauth-serving-cert\") pod \"console-75449b9c89-dwplx\" (UID: \"17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77\") " pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:38.861498 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.861441    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77-console-config\") pod \"console-75449b9c89-dwplx\" (UID: \"17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77\") " pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:38.862369 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.862345    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77-trusted-ca-bundle\") pod \"console-75449b9c89-dwplx\" (UID: \"17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77\") " pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:38.863143 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.863120    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77-console-serving-cert\") pod \"console-75449b9c89-dwplx\" (UID: \"17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77\") " pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:38.863256 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.863239    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77-console-oauth-config\") pod \"console-75449b9c89-dwplx\" (UID: \"17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77\") " pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:38.872208 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.872177    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbz4m\" (UniqueName: \"kubernetes.io/projected/71d7f7c4-a64e-4aa9-95bf-aa89365595c6-kube-api-access-vbz4m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-42rjm\" (UID: \"71d7f7c4-a64e-4aa9-95bf-aa89365595c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm"
Jul 30 17:38:38.872337 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.872320    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lq4v\" (UniqueName: \"kubernetes.io/projected/17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77-kube-api-access-7lq4v\") pod \"console-75449b9c89-dwplx\" (UID: \"17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77\") " pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:38.949213 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:38.949182    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:39.023471 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:39.023436    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:39.040605 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:39.040572    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm"
Jul 30 17:38:39.171738 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:39.171704    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75449b9c89-dwplx"]
Jul 30 17:38:39.175336 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:38:39.175311    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17bfb507_2a6d_4cfe_a87a_2c2d53fb8b77.slice/crio-080458557e585bbeb51bc81d86679e8c842405e183b07245ea9c7fc426110b8b WatchSource:0}: Error finding container 080458557e585bbeb51bc81d86679e8c842405e183b07245ea9c7fc426110b8b: Status 404 returned error can't find the container with id 080458557e585bbeb51bc81d86679e8c842405e183b07245ea9c7fc426110b8b
Jul 30 17:38:39.200838 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:39.197049    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm"]
Jul 30 17:38:39.348126 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:39.348040    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:39.820917 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:39.820874    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75449b9c89-dwplx" event={"ID":"17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77","Type":"ContainerStarted","Data":"8c5780abf7041966638c683ca5d54e421f2d84926ce8e507e8165736127ff898"}
Jul 30 17:38:39.821370 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:39.820931    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75449b9c89-dwplx" event={"ID":"17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77","Type":"ContainerStarted","Data":"080458557e585bbeb51bc81d86679e8c842405e183b07245ea9c7fc426110b8b"}
Jul 30 17:38:39.822459 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:39.822432    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm" event={"ID":"71d7f7c4-a64e-4aa9-95bf-aa89365595c6","Type":"ContainerStarted","Data":"7ee55d96ae43843d437a3da937df84266ea2a1458174029fb03e1ad18ab2be6e"}
Jul 30 17:38:39.843996 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:39.843940    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75449b9c89-dwplx" podStartSLOduration=1.843924727 podStartE2EDuration="1.843924727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:38:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:38:39.842256447 +0000 UTC m=+694.295820451" watchObservedRunningTime="2026-07-30 17:38:39.843924727 +0000 UTC m=+694.297488731"
Jul 30 17:38:40.677142 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:40.677103    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lsjt5"]
Jul 30 17:38:40.681183 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:40.681151    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lsjt5"
Jul 30 17:38:40.684053 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:40.684032    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-l7hnj\""
Jul 30 17:38:40.696555 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:40.696528    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lsjt5"]
Jul 30 17:38:40.777860 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:40.777796    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzxx5\" (UniqueName: \"kubernetes.io/projected/5ec97919-a10c-4782-a193-150e60fd68b0-kube-api-access-zzxx5\") pod \"limitador-operator-controller-manager-859d7b78bd-lsjt5\" (UID: \"5ec97919-a10c-4782-a193-150e60fd68b0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lsjt5"
Jul 30 17:38:40.879213 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:40.879174    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzxx5\" (UniqueName: \"kubernetes.io/projected/5ec97919-a10c-4782-a193-150e60fd68b0-kube-api-access-zzxx5\") pod \"limitador-operator-controller-manager-859d7b78bd-lsjt5\" (UID: \"5ec97919-a10c-4782-a193-150e60fd68b0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lsjt5"
Jul 30 17:38:40.894060 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:40.894030    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzxx5\" (UniqueName: \"kubernetes.io/projected/5ec97919-a10c-4782-a193-150e60fd68b0-kube-api-access-zzxx5\") pod \"limitador-operator-controller-manager-859d7b78bd-lsjt5\" (UID: \"5ec97919-a10c-4782-a193-150e60fd68b0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lsjt5"
Jul 30 17:38:40.935244 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:40.935172    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:40.993768 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:40.993733    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lsjt5"
Jul 30 17:38:41.133856 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:41.133834    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lsjt5"]
Jul 30 17:38:41.135654 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:38:41.135632    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ec97919_a10c_4782_a193_150e60fd68b0.slice/crio-984a6ae6dedef027ef3f29dbd5f646dfc93700d83c8b4d4c39ac054fcdff557f WatchSource:0}: Error finding container 984a6ae6dedef027ef3f29dbd5f646dfc93700d83c8b4d4c39ac054fcdff557f: Status 404 returned error can't find the container with id 984a6ae6dedef027ef3f29dbd5f646dfc93700d83c8b4d4c39ac054fcdff557f
Jul 30 17:38:41.335934 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:41.335896    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:41.835121 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:41.835084    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lsjt5" event={"ID":"5ec97919-a10c-4782-a193-150e60fd68b0","Type":"ContainerStarted","Data":"984a6ae6dedef027ef3f29dbd5f646dfc93700d83c8b4d4c39ac054fcdff557f"}
Jul 30 17:38:42.783939 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:42.783902    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:43.198276 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:43.198189    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-vgzwj"]
Jul 30 17:38:43.201588 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:43.201569    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vgzwj"
Jul 30 17:38:43.203788 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:43.203770    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-kbbck\""
Jul 30 17:38:43.211055 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:43.211034    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-vgzwj"]
Jul 30 17:38:43.302712 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:43.302665    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9cpz\" (UniqueName: \"kubernetes.io/projected/a5d7663e-d9ab-410e-9759-7477ba5c5f90-kube-api-access-t9cpz\") pod \"authorino-operator-75cc8d5cb7-vgzwj\" (UID: \"a5d7663e-d9ab-410e-9759-7477ba5c5f90\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-vgzwj"
Jul 30 17:38:43.404060 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:43.404023    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9cpz\" (UniqueName: \"kubernetes.io/projected/a5d7663e-d9ab-410e-9759-7477ba5c5f90-kube-api-access-t9cpz\") pod \"authorino-operator-75cc8d5cb7-vgzwj\" (UID: \"a5d7663e-d9ab-410e-9759-7477ba5c5f90\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-vgzwj"
Jul 30 17:38:43.416515 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:43.416481    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9cpz\" (UniqueName: \"kubernetes.io/projected/a5d7663e-d9ab-410e-9759-7477ba5c5f90-kube-api-access-t9cpz\") pod \"authorino-operator-75cc8d5cb7-vgzwj\" (UID: \"a5d7663e-d9ab-410e-9759-7477ba5c5f90\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-vgzwj"
Jul 30 17:38:43.512257 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:43.512224    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vgzwj"
Jul 30 17:38:43.846571 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:43.846480    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lsjt5" event={"ID":"5ec97919-a10c-4782-a193-150e60fd68b0","Type":"ContainerStarted","Data":"cd41601ad14022a8138b069262c3ec98ffe53153c27e23e8715b51a8a1593a43"}
Jul 30 17:38:43.846571 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:43.846561    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lsjt5"
Jul 30 17:38:43.876081 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:43.876018    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lsjt5" podStartSLOduration=2.144585726 podStartE2EDuration="3.875996186s" totalImagesPullingTime="1.73141046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:38:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:38:43.872769623 +0000 UTC m=+698.326333630" watchObservedRunningTime="2026-07-30 17:38:43.875996186 +0000 UTC m=+698.329560178"
Jul 30 17:38:44.358981 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:44.358947    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:44.735671 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:44.735596    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-vgzwj"]
Jul 30 17:38:44.738204 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:38:44.738169    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5d7663e_d9ab_410e_9759_7477ba5c5f90.slice/crio-424f88c8c0af6104efdc3ccb65bebb6f445da1cf9dcd42e84da6d48cdddfe332 WatchSource:0}: Error finding container 424f88c8c0af6104efdc3ccb65bebb6f445da1cf9dcd42e84da6d48cdddfe332: Status 404 returned error can't find the container with id 424f88c8c0af6104efdc3ccb65bebb6f445da1cf9dcd42e84da6d48cdddfe332
Jul 30 17:38:44.852452 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:44.852414    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm" event={"ID":"71d7f7c4-a64e-4aa9-95bf-aa89365595c6","Type":"ContainerStarted","Data":"b337fdc1d63a3062c8a158743c38bcfd31f035c546457b82f3205cadbf7dfc7a"}
Jul 30 17:38:44.852922 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:44.852511    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm"
Jul 30 17:38:44.853700 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:44.853679    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vgzwj" event={"ID":"a5d7663e-d9ab-410e-9759-7477ba5c5f90","Type":"ContainerStarted","Data":"424f88c8c0af6104efdc3ccb65bebb6f445da1cf9dcd42e84da6d48cdddfe332"}
Jul 30 17:38:44.880226 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:44.880168    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm" podStartSLOduration=1.41866243 podStartE2EDuration="6.88014855s" totalImagesPullingTime="5.46148612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:38:44.879017585 +0000 UTC m=+699.332581558" watchObservedRunningTime="2026-07-30 17:38:44.88014855 +0000 UTC m=+699.333712542"
Jul 30 17:38:45.194025 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:45.193988    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:45.292786 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:45.292747    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:46.558162 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:46.558117    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lblst"]
Jul 30 17:38:46.569593 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:46.569558    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lblst"
Jul 30 17:38:46.572156 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:46.572123    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 30 17:38:46.572392 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:46.572375    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-4tgn2\""
Jul 30 17:38:46.573918 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:46.573488    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lblst"]
Jul 30 17:38:46.635737 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:46.635703    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjrmr\" (UniqueName: \"kubernetes.io/projected/2c73c8dc-987b-4121-88f7-c7a90683936a-kube-api-access-wjrmr\") pod \"dns-operator-controller-manager-864dd78c4c-lblst\" (UID: \"2c73c8dc-987b-4121-88f7-c7a90683936a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lblst"
Jul 30 17:38:46.736461 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:46.736429    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjrmr\" (UniqueName: \"kubernetes.io/projected/2c73c8dc-987b-4121-88f7-c7a90683936a-kube-api-access-wjrmr\") pod \"dns-operator-controller-manager-864dd78c4c-lblst\" (UID: \"2c73c8dc-987b-4121-88f7-c7a90683936a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lblst"
Jul 30 17:38:46.751766 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:46.751732    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjrmr\" (UniqueName: \"kubernetes.io/projected/2c73c8dc-987b-4121-88f7-c7a90683936a-kube-api-access-wjrmr\") pod \"dns-operator-controller-manager-864dd78c4c-lblst\" (UID: \"2c73c8dc-987b-4121-88f7-c7a90683936a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lblst"
Jul 30 17:38:46.884096 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:46.884002    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lblst"
Jul 30 17:38:47.031808 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:47.031776    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lblst"]
Jul 30 17:38:47.232963 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:38:47.232888    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c73c8dc_987b_4121_88f7_c7a90683936a.slice/crio-ef1f46dfb77e36b6c121508813ada567d3b818c2a3138fc2b621bd3363344c56 WatchSource:0}: Error finding container ef1f46dfb77e36b6c121508813ada567d3b818c2a3138fc2b621bd3363344c56: Status 404 returned error can't find the container with id ef1f46dfb77e36b6c121508813ada567d3b818c2a3138fc2b621bd3363344c56
Jul 30 17:38:47.239635 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:47.239597    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:47.765218 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:47.765187    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:47.847780 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:47.847750    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:47.873609 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:47.873571    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vgzwj" event={"ID":"a5d7663e-d9ab-410e-9759-7477ba5c5f90","Type":"ContainerStarted","Data":"bce3aaa5ed1d623040d27440419889256049df499e5df3ee5a25ebe82b99872b"}
Jul 30 17:38:47.873837 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:47.873792    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vgzwj"
Jul 30 17:38:47.875068 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:47.875041    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lblst" event={"ID":"2c73c8dc-987b-4121-88f7-c7a90683936a","Type":"ContainerStarted","Data":"ef1f46dfb77e36b6c121508813ada567d3b818c2a3138fc2b621bd3363344c56"}
Jul 30 17:38:47.894370 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:47.894324    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vgzwj" podStartSLOduration=2.352032161 podStartE2EDuration="4.894309215s" totalImagesPullingTime="2.542277054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:38:47.89265191 +0000 UTC m=+702.346215903" watchObservedRunningTime="2026-07-30 17:38:47.894309215 +0000 UTC m=+702.347873206"
Jul 30 17:38:49.024423 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:49.024384    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:49.024878 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:49.024453    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:49.031551 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:49.031523    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:49.154523 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:49.154482    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:49.255228 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:49.255198    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:49.886398 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:49.886361    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lblst" event={"ID":"2c73c8dc-987b-4121-88f7-c7a90683936a","Type":"ContainerStarted","Data":"dd5ead4fef021e013d3cee71a7e09c9498f9045e6970ec79a5fd53c14c1ea50c"}
Jul 30 17:38:49.886594 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:49.886575    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lblst"
Jul 30 17:38:49.890483 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:49.890463    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75449b9c89-dwplx"
Jul 30 17:38:49.909156 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:49.909099    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lblst" podStartSLOduration=1.8616140159999999 podStartE2EDuration="3.909081846s" totalImagesPullingTime="2.04746783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:38:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:38:49.908951908 +0000 UTC m=+704.362515898" watchObservedRunningTime="2026-07-30 17:38:49.909081846 +0000 UTC m=+704.362645840"
Jul 30 17:38:50.099481 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:50.099446    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:50.200058 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:50.199969    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:50.269145 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:50.269116    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:54.856752 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:54.856725    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lsjt5"
Jul 30 17:38:55.864455 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:55.864424    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm"
Jul 30 17:38:56.621445 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:56.621412    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:56.738105 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:56.738071    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:57.710774 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.710743    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8lxk"]
Jul 30 17:38:57.714248 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.714234    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8lxk"
Jul 30 17:38:57.728162 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.728139    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8lxk"]
Jul 30 17:38:57.791222 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.791191    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm"]
Jul 30 17:38:57.791391 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.791368    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm" podUID="71d7f7c4-a64e-4aa9-95bf-aa89365595c6" containerName="manager" containerID="cri-o://b337fdc1d63a3062c8a158743c38bcfd31f035c546457b82f3205cadbf7dfc7a" gracePeriod=2
Jul 30 17:38:57.800217 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.800189    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm"]
Jul 30 17:38:57.812661 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.812637    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8lxk"]
Jul 30 17:38:57.812955 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:38:57.812936    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-jhqfq], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8lxk" podUID="12428b45-1247-471a-85ea-95a1a5b77778"
Jul 30 17:38:57.824066 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.824042    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8lxk"]
Jul 30 17:38:57.839391 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.839367    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhqfq\" (UniqueName: \"kubernetes.io/projected/12428b45-1247-471a-85ea-95a1a5b77778-kube-api-access-jhqfq\") pod \"kuadrant-operator-controller-manager-86878c769f-p8lxk\" (UID: \"12428b45-1247-471a-85ea-95a1a5b77778\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8lxk"
Jul 30 17:38:57.839511 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.839419    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12428b45-1247-471a-85ea-95a1a5b77778-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-p8lxk\" (UID: \"12428b45-1247-471a-85ea-95a1a5b77778\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8lxk"
Jul 30 17:38:57.844389 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.844367    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t"]
Jul 30 17:38:57.844711 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.844700    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71d7f7c4-a64e-4aa9-95bf-aa89365595c6" containerName="manager"
Jul 30 17:38:57.844748 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.844713    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="71d7f7c4-a64e-4aa9-95bf-aa89365595c6" containerName="manager"
Jul 30 17:38:57.844787 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.844778    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="71d7f7c4-a64e-4aa9-95bf-aa89365595c6" containerName="manager"
Jul 30 17:38:57.847800 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.847780    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t"
Jul 30 17:38:57.848638 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.848619    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lsjt5"]
Jul 30 17:38:57.848805 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.848785    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lsjt5" podUID="5ec97919-a10c-4782-a193-150e60fd68b0" containerName="manager" containerID="cri-o://cd41601ad14022a8138b069262c3ec98ffe53153c27e23e8715b51a8a1593a43" gracePeriod=2
Jul 30 17:38:57.858010 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.857988    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lsjt5"]
Jul 30 17:38:57.865620 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.865586    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t"]
Jul 30 17:38:57.880333 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.880306    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sjv69"]
Jul 30 17:38:57.880707 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.880692    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ec97919-a10c-4782-a193-150e60fd68b0" containerName="manager"
Jul 30 17:38:57.880778 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.880710    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ec97919-a10c-4782-a193-150e60fd68b0" containerName="manager"
Jul 30 17:38:57.880860 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.880840    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ec97919-a10c-4782-a193-150e60fd68b0" containerName="manager"
Jul 30 17:38:57.883897 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.883877    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sjv69"
Jul 30 17:38:57.888032 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:38:57.888002    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-42rjm\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" podUID="71d7f7c4-a64e-4aa9-95bf-aa89365595c6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm"
Jul 30 17:38:57.896390 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.896367    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sjv69"]
Jul 30 17:38:57.918350 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:38:57.918313    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-42rjm\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" podUID="71d7f7c4-a64e-4aa9-95bf-aa89365595c6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm"
Jul 30 17:38:57.920046 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:38:57.920024    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-lsjt5\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" podUID="5ec97919-a10c-4782-a193-150e60fd68b0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lsjt5"
Jul 30 17:38:57.926839 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.926795    2577 generic.go:356] "Generic (PLEG): container finished" podID="71d7f7c4-a64e-4aa9-95bf-aa89365595c6" containerID="b337fdc1d63a3062c8a158743c38bcfd31f035c546457b82f3205cadbf7dfc7a" exitCode=0
Jul 30 17:38:57.926947 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.926912    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8lxk"
Jul 30 17:38:57.929089 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:38:57.929064    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-lsjt5\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" podUID="5ec97919-a10c-4782-a193-150e60fd68b0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lsjt5"
Jul 30 17:38:57.930809 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:38:57.930790    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-42rjm\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" podUID="71d7f7c4-a64e-4aa9-95bf-aa89365595c6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm"
Jul 30 17:38:57.932546 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:38:57.932516    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-p8lxk\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" podUID="12428b45-1247-471a-85ea-95a1a5b77778" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8lxk"
Jul 30 17:38:57.940006 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.939981    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhqfq\" (UniqueName: \"kubernetes.io/projected/12428b45-1247-471a-85ea-95a1a5b77778-kube-api-access-jhqfq\") pod \"kuadrant-operator-controller-manager-86878c769f-p8lxk\" (UID: \"12428b45-1247-471a-85ea-95a1a5b77778\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8lxk"
Jul 30 17:38:57.940117 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.940018    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bgpm\" (UniqueName: \"kubernetes.io/projected/1d59e94d-29f3-48ba-9ae8-d8d0de63e124-kube-api-access-4bgpm\") pod \"kuadrant-operator-controller-manager-86878c769f-xhr6t\" (UID: \"1d59e94d-29f3-48ba-9ae8-d8d0de63e124\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t"
Jul 30 17:38:57.940117 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.940076    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d59e94d-29f3-48ba-9ae8-d8d0de63e124-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xhr6t\" (UID: \"1d59e94d-29f3-48ba-9ae8-d8d0de63e124\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t"
Jul 30 17:38:57.940117 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.940115    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12428b45-1247-471a-85ea-95a1a5b77778-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-p8lxk\" (UID: \"12428b45-1247-471a-85ea-95a1a5b77778\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8lxk"
Jul 30 17:38:57.940250 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.940204    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwfdd\" (UniqueName: \"kubernetes.io/projected/73e3ca24-3939-4b09-89af-01b8100030cd-kube-api-access-cwfdd\") pod \"limitador-operator-controller-manager-859d7b78bd-sjv69\" (UID: \"73e3ca24-3939-4b09-89af-01b8100030cd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sjv69"
Jul 30 17:38:57.940388 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.940373    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12428b45-1247-471a-85ea-95a1a5b77778-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-p8lxk\" (UID: \"12428b45-1247-471a-85ea-95a1a5b77778\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8lxk"
Jul 30 17:38:57.942845 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:57.942809    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8lxk"
Jul 30 17:38:57.944420 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:38:57.944405    2577 projected.go:196] Error preparing data for projected volume kube-api-access-jhqfq for pod kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8lxk: failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-p8lxk" not found
Jul 30 17:38:57.944494 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:38:57.944464    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12428b45-1247-471a-85ea-95a1a5b77778-kube-api-access-jhqfq podName:12428b45-1247-471a-85ea-95a1a5b77778 nodeName:}" failed. No retries permitted until 2026-07-30 17:38:58.444450638 +0000 UTC m=+712.898014607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jhqfq" (UniqueName: "kubernetes.io/projected/12428b45-1247-471a-85ea-95a1a5b77778-kube-api-access-jhqfq") pod "kuadrant-operator-controller-manager-86878c769f-p8lxk" (UID: "12428b45-1247-471a-85ea-95a1a5b77778") : failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-p8lxk" not found
Jul 30 17:38:57.944737 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:38:57.944717    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-p8lxk\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" podUID="12428b45-1247-471a-85ea-95a1a5b77778" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8lxk"
Jul 30 17:38:57.946054 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:38:57.946031    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-lsjt5\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" podUID="5ec97919-a10c-4782-a193-150e60fd68b0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lsjt5"
Jul 30 17:38:57.947435 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:38:57.947414    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-42rjm\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" podUID="71d7f7c4-a64e-4aa9-95bf-aa89365595c6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm"
Jul 30 17:38:58.041364 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.041331    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12428b45-1247-471a-85ea-95a1a5b77778-extensions-socket-volume\") pod \"12428b45-1247-471a-85ea-95a1a5b77778\" (UID: \"12428b45-1247-471a-85ea-95a1a5b77778\") "
Jul 30 17:38:58.041509 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.041495    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwfdd\" (UniqueName: \"kubernetes.io/projected/73e3ca24-3939-4b09-89af-01b8100030cd-kube-api-access-cwfdd\") pod \"limitador-operator-controller-manager-859d7b78bd-sjv69\" (UID: \"73e3ca24-3939-4b09-89af-01b8100030cd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sjv69"
Jul 30 17:38:58.041602 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.041581    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bgpm\" (UniqueName: \"kubernetes.io/projected/1d59e94d-29f3-48ba-9ae8-d8d0de63e124-kube-api-access-4bgpm\") pod \"kuadrant-operator-controller-manager-86878c769f-xhr6t\" (UID: \"1d59e94d-29f3-48ba-9ae8-d8d0de63e124\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t"
Jul 30 17:38:58.041717 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.041579    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12428b45-1247-471a-85ea-95a1a5b77778-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "12428b45-1247-471a-85ea-95a1a5b77778" (UID: "12428b45-1247-471a-85ea-95a1a5b77778"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:38:58.041717 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.041646    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d59e94d-29f3-48ba-9ae8-d8d0de63e124-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xhr6t\" (UID: \"1d59e94d-29f3-48ba-9ae8-d8d0de63e124\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t"
Jul 30 17:38:58.041845 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.041747    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12428b45-1247-471a-85ea-95a1a5b77778-extensions-socket-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:38:58.041845 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.041766    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhqfq\" (UniqueName: \"kubernetes.io/projected/12428b45-1247-471a-85ea-95a1a5b77778-kube-api-access-jhqfq\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:38:58.042025 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.042005    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d59e94d-29f3-48ba-9ae8-d8d0de63e124-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xhr6t\" (UID: \"1d59e94d-29f3-48ba-9ae8-d8d0de63e124\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t"
Jul 30 17:38:58.053504 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.053476    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwfdd\" (UniqueName: \"kubernetes.io/projected/73e3ca24-3939-4b09-89af-01b8100030cd-kube-api-access-cwfdd\") pod \"limitador-operator-controller-manager-859d7b78bd-sjv69\" (UID: \"73e3ca24-3939-4b09-89af-01b8100030cd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sjv69"
Jul 30 17:38:58.053632 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.053484    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bgpm\" (UniqueName: \"kubernetes.io/projected/1d59e94d-29f3-48ba-9ae8-d8d0de63e124-kube-api-access-4bgpm\") pod \"kuadrant-operator-controller-manager-86878c769f-xhr6t\" (UID: \"1d59e94d-29f3-48ba-9ae8-d8d0de63e124\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t"
Jul 30 17:38:58.087408 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.087381    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lsjt5"
Jul 30 17:38:58.089539 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.089518    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm"
Jul 30 17:38:58.089631 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:38:58.089536    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-p8lxk\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" podUID="12428b45-1247-471a-85ea-95a1a5b77778" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8lxk"
Jul 30 17:38:58.091172 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:38:58.091153    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-lsjt5\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" podUID="5ec97919-a10c-4782-a193-150e60fd68b0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lsjt5"
Jul 30 17:38:58.092837 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:38:58.092798    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-42rjm\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" podUID="71d7f7c4-a64e-4aa9-95bf-aa89365595c6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm"
Jul 30 17:38:58.094451 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:38:58.094429    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-p8lxk\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" podUID="12428b45-1247-471a-85ea-95a1a5b77778" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8lxk"
Jul 30 17:38:58.095872 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:38:58.095851    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-lsjt5\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" podUID="5ec97919-a10c-4782-a193-150e60fd68b0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lsjt5"
Jul 30 17:38:58.097150 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:38:58.097129    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-42rjm\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" podUID="71d7f7c4-a64e-4aa9-95bf-aa89365595c6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm"
Jul 30 17:38:58.110359 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.110337    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12428b45-1247-471a-85ea-95a1a5b77778" path="/var/lib/kubelet/pods/12428b45-1247-471a-85ea-95a1a5b77778/volumes"
Jul 30 17:38:58.233078 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.232989    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t"
Jul 30 17:38:58.239912 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.239893    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sjv69"
Jul 30 17:38:58.243936 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.243688    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71d7f7c4-a64e-4aa9-95bf-aa89365595c6-extensions-socket-volume\") pod \"71d7f7c4-a64e-4aa9-95bf-aa89365595c6\" (UID: \"71d7f7c4-a64e-4aa9-95bf-aa89365595c6\") "
Jul 30 17:38:58.243936 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.243728    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzxx5\" (UniqueName: \"kubernetes.io/projected/5ec97919-a10c-4782-a193-150e60fd68b0-kube-api-access-zzxx5\") pod \"5ec97919-a10c-4782-a193-150e60fd68b0\" (UID: \"5ec97919-a10c-4782-a193-150e60fd68b0\") "
Jul 30 17:38:58.243936 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.243895    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbz4m\" (UniqueName: \"kubernetes.io/projected/71d7f7c4-a64e-4aa9-95bf-aa89365595c6-kube-api-access-vbz4m\") pod \"71d7f7c4-a64e-4aa9-95bf-aa89365595c6\" (UID: \"71d7f7c4-a64e-4aa9-95bf-aa89365595c6\") "
Jul 30 17:38:58.244192 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.244165    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d7f7c4-a64e-4aa9-95bf-aa89365595c6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "71d7f7c4-a64e-4aa9-95bf-aa89365595c6" (UID: "71d7f7c4-a64e-4aa9-95bf-aa89365595c6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:38:58.246007 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.245977    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ec97919-a10c-4782-a193-150e60fd68b0-kube-api-access-zzxx5" (OuterVolumeSpecName: "kube-api-access-zzxx5") pod "5ec97919-a10c-4782-a193-150e60fd68b0" (UID: "5ec97919-a10c-4782-a193-150e60fd68b0"). InnerVolumeSpecName "kube-api-access-zzxx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:38:58.246007 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.245993    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71d7f7c4-a64e-4aa9-95bf-aa89365595c6-kube-api-access-vbz4m" (OuterVolumeSpecName: "kube-api-access-vbz4m") pod "71d7f7c4-a64e-4aa9-95bf-aa89365595c6" (UID: "71d7f7c4-a64e-4aa9-95bf-aa89365595c6"). InnerVolumeSpecName "kube-api-access-vbz4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:38:58.345325 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.345292    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbz4m\" (UniqueName: \"kubernetes.io/projected/71d7f7c4-a64e-4aa9-95bf-aa89365595c6-kube-api-access-vbz4m\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:38:58.345325 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.345329    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71d7f7c4-a64e-4aa9-95bf-aa89365595c6-extensions-socket-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:38:58.345506 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.345343    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzxx5\" (UniqueName: \"kubernetes.io/projected/5ec97919-a10c-4782-a193-150e60fd68b0-kube-api-access-zzxx5\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:38:58.399940 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.399915    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t"]
Jul 30 17:38:58.402405 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:38:58.402381    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d59e94d_29f3_48ba_9ae8_d8d0de63e124.slice/crio-5e59fe545b8fc22a05f23fd315d55737f537e7fb42a9a813960fb348de31c7bc WatchSource:0}: Error finding container 5e59fe545b8fc22a05f23fd315d55737f537e7fb42a9a813960fb348de31c7bc: Status 404 returned error can't find the container with id 5e59fe545b8fc22a05f23fd315d55737f537e7fb42a9a813960fb348de31c7bc
Jul 30 17:38:58.405021 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.404997    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sjv69"]
Jul 30 17:38:58.407288 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:38:58.407266    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73e3ca24_3939_4b09_89af_01b8100030cd.slice/crio-c074b8b8e7e57dcd40cec7751cb9097307e2970897e3899f344981c62a189e90 WatchSource:0}: Error finding container c074b8b8e7e57dcd40cec7751cb9097307e2970897e3899f344981c62a189e90: Status 404 returned error can't find the container with id c074b8b8e7e57dcd40cec7751cb9097307e2970897e3899f344981c62a189e90
Jul 30 17:38:58.722686 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.722612    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:58.827073 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.827046    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:38:58.882781 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.882755    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vgzwj"
Jul 30 17:38:58.933369 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.933341    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-42rjm"
Jul 30 17:38:58.933369 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.933342    2577 scope.go:117] "RemoveContainer" containerID="b337fdc1d63a3062c8a158743c38bcfd31f035c546457b82f3205cadbf7dfc7a"
Jul 30 17:38:58.936404 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.936372    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sjv69" event={"ID":"73e3ca24-3939-4b09-89af-01b8100030cd","Type":"ContainerStarted","Data":"fd3e7dc60b773c002281c3586d90519814d455a159a66432f3ec3ca1a63465b6"}
Jul 30 17:38:58.936877 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.936857    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sjv69"
Jul 30 17:38:58.936987 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.936902    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sjv69" event={"ID":"73e3ca24-3939-4b09-89af-01b8100030cd","Type":"ContainerStarted","Data":"c074b8b8e7e57dcd40cec7751cb9097307e2970897e3899f344981c62a189e90"}
Jul 30 17:38:58.938496 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.938473    2577 generic.go:356] "Generic (PLEG): container finished" podID="5ec97919-a10c-4782-a193-150e60fd68b0" containerID="cd41601ad14022a8138b069262c3ec98ffe53153c27e23e8715b51a8a1593a43" exitCode=0
Jul 30 17:38:58.938622 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.938608    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lsjt5"
Jul 30 17:38:58.940779 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.940761    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8lxk"
Jul 30 17:38:58.941316 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.941154    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t" event={"ID":"1d59e94d-29f3-48ba-9ae8-d8d0de63e124","Type":"ContainerStarted","Data":"f53ffa6dd3b6161a4a4a40ed4464f20909923b8de54ae3830cb4d735b291a657"}
Jul 30 17:38:58.941316 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.941205    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t"
Jul 30 17:38:58.941316 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.941230    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t" event={"ID":"1d59e94d-29f3-48ba-9ae8-d8d0de63e124","Type":"ContainerStarted","Data":"5e59fe545b8fc22a05f23fd315d55737f537e7fb42a9a813960fb348de31c7bc"}
Jul 30 17:38:58.946281 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.946259    2577 scope.go:117] "RemoveContainer" containerID="cd41601ad14022a8138b069262c3ec98ffe53153c27e23e8715b51a8a1593a43"
Jul 30 17:38:58.955614 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.955599    2577 scope.go:117] "RemoveContainer" containerID="cd41601ad14022a8138b069262c3ec98ffe53153c27e23e8715b51a8a1593a43"
Jul 30 17:38:58.956010 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:38:58.955969    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd41601ad14022a8138b069262c3ec98ffe53153c27e23e8715b51a8a1593a43\": container with ID starting with cd41601ad14022a8138b069262c3ec98ffe53153c27e23e8715b51a8a1593a43 not found: ID does not exist" containerID="cd41601ad14022a8138b069262c3ec98ffe53153c27e23e8715b51a8a1593a43"
Jul 30 17:38:58.956103 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.956011    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd41601ad14022a8138b069262c3ec98ffe53153c27e23e8715b51a8a1593a43"} err="failed to get container status \"cd41601ad14022a8138b069262c3ec98ffe53153c27e23e8715b51a8a1593a43\": rpc error: code = NotFound desc = could not find container \"cd41601ad14022a8138b069262c3ec98ffe53153c27e23e8715b51a8a1593a43\": container with ID starting with cd41601ad14022a8138b069262c3ec98ffe53153c27e23e8715b51a8a1593a43 not found: ID does not exist"
Jul 30 17:38:58.967394 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.967346    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sjv69" podStartSLOduration=1.96733066 podStartE2EDuration="1.96733066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:38:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:38:58.966365615 +0000 UTC m=+713.419929607" watchObservedRunningTime="2026-07-30 17:38:58.96733066 +0000 UTC m=+713.420894651"
Jul 30 17:38:58.985481 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:38:58.985398    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t" podStartSLOduration=1.9853837680000002 podStartE2EDuration="1.985383768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:38:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:38:58.984240948 +0000 UTC m=+713.437804939" watchObservedRunningTime="2026-07-30 17:38:58.985383768 +0000 UTC m=+713.438947759"
Jul 30 17:38:58.986141 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:38:58.986114    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-p8lxk\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" podUID="12428b45-1247-471a-85ea-95a1a5b77778" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8lxk"
Jul 30 17:39:00.110472 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:00.110439    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ec97919-a10c-4782-a193-150e60fd68b0" path="/var/lib/kubelet/pods/5ec97919-a10c-4782-a193-150e60fd68b0/volumes"
Jul 30 17:39:00.110848 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:00.110776    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71d7f7c4-a64e-4aa9-95bf-aa89365595c6" path="/var/lib/kubelet/pods/71d7f7c4-a64e-4aa9-95bf-aa89365595c6/volumes"
Jul 30 17:39:00.893914 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:00.893886    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lblst"
Jul 30 17:39:01.364716 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:01.364684    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:01.469379 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:01.469346    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:03.802737 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:03.802700    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:03.900203 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:03.900175    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:09.355243 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:09.355212    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:09.456295 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:09.456263    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:09.949881 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:09.949854    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t"
Jul 30 17:39:09.950045 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:09.949924    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sjv69"
Jul 30 17:39:16.286599 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:16.286563    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:16.396263 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:16.396228    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:21.096580 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:21.096547    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:21.197921 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:21.197883    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:23.549270 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:23.549235    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:23.649940 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:23.649905    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:26.415533 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:26.415495    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:26.673214 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:26.673125    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:28.773352 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:28.773315    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:28.784638 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:28.784606    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-9s996"]
Jul 30 17:39:28.787062 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:28.787044    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-9s996"
Jul 30 17:39:28.789441 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:28.789423    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 30 17:39:28.789534 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:28.789465    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wcchl\""
Jul 30 17:39:28.796443 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:28.796420    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-9s996"]
Jul 30 17:39:28.814146 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:28.814115    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmnwk\" (UniqueName: \"kubernetes.io/projected/e7935de7-4494-46f8-ab26-5636fd38d25e-kube-api-access-vmnwk\") pod \"limitador-limitador-64c94fd64f-9s996\" (UID: \"e7935de7-4494-46f8-ab26-5636fd38d25e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-9s996"
Jul 30 17:39:28.814302 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:28.814155    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7935de7-4494-46f8-ab26-5636fd38d25e-config-file\") pod \"limitador-limitador-64c94fd64f-9s996\" (UID: \"e7935de7-4494-46f8-ab26-5636fd38d25e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-9s996"
Jul 30 17:39:28.880711 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:28.880676    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-9s996"]
Jul 30 17:39:28.891595 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:28.891559    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:28.915121 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:28.915094    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmnwk\" (UniqueName: \"kubernetes.io/projected/e7935de7-4494-46f8-ab26-5636fd38d25e-kube-api-access-vmnwk\") pod \"limitador-limitador-64c94fd64f-9s996\" (UID: \"e7935de7-4494-46f8-ab26-5636fd38d25e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-9s996"
Jul 30 17:39:28.915283 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:28.915133    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7935de7-4494-46f8-ab26-5636fd38d25e-config-file\") pod \"limitador-limitador-64c94fd64f-9s996\" (UID: \"e7935de7-4494-46f8-ab26-5636fd38d25e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-9s996"
Jul 30 17:39:28.915685 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:28.915667    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7935de7-4494-46f8-ab26-5636fd38d25e-config-file\") pod \"limitador-limitador-64c94fd64f-9s996\" (UID: \"e7935de7-4494-46f8-ab26-5636fd38d25e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-9s996"
Jul 30 17:39:28.924922 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:28.924897    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmnwk\" (UniqueName: \"kubernetes.io/projected/e7935de7-4494-46f8-ab26-5636fd38d25e-kube-api-access-vmnwk\") pod \"limitador-limitador-64c94fd64f-9s996\" (UID: \"e7935de7-4494-46f8-ab26-5636fd38d25e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-9s996"
Jul 30 17:39:28.992094 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:28.992064    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:29.097776 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:29.097697    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-9s996"
Jul 30 17:39:29.239289 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:29.239255    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-9s996"]
Jul 30 17:39:29.242370 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:39:29.242328    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7935de7_4494_46f8_ab26_5636fd38d25e.slice/crio-a650dab1ccbf3f1ce94de0a373669288205d4be03d7e5f971a6030840367d41c WatchSource:0}: Error finding container a650dab1ccbf3f1ce94de0a373669288205d4be03d7e5f971a6030840367d41c: Status 404 returned error can't find the container with id a650dab1ccbf3f1ce94de0a373669288205d4be03d7e5f971a6030840367d41c
Jul 30 17:39:29.311392 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:29.311356    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:29.384637 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:29.384554    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:29.583286 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:29.583256    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-q7tt6"]
Jul 30 17:39:29.586594 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:29.586560    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-q7tt6"
Jul 30 17:39:29.589177 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:29.589132    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-d8xl2\""
Jul 30 17:39:29.596917 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:29.596894    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-q7tt6"]
Jul 30 17:39:29.620827 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:29.620784    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnqzx\" (UniqueName: \"kubernetes.io/projected/4338b036-d2fa-4e35-8238-a878f3223ba4-kube-api-access-tnqzx\") pod \"authorino-58cd955675-q7tt6\" (UID: \"4338b036-d2fa-4e35-8238-a878f3223ba4\") " pod="kuadrant-system/authorino-58cd955675-q7tt6"
Jul 30 17:39:29.626627 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:29.626592    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:29.705019 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:29.704941    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:29.722140 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:29.722111    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnqzx\" (UniqueName: \"kubernetes.io/projected/4338b036-d2fa-4e35-8238-a878f3223ba4-kube-api-access-tnqzx\") pod \"authorino-58cd955675-q7tt6\" (UID: \"4338b036-d2fa-4e35-8238-a878f3223ba4\") " pod="kuadrant-system/authorino-58cd955675-q7tt6"
Jul 30 17:39:29.730617 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:29.730591    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnqzx\" (UniqueName: \"kubernetes.io/projected/4338b036-d2fa-4e35-8238-a878f3223ba4-kube-api-access-tnqzx\") pod \"authorino-58cd955675-q7tt6\" (UID: \"4338b036-d2fa-4e35-8238-a878f3223ba4\") " pod="kuadrant-system/authorino-58cd955675-q7tt6"
Jul 30 17:39:29.897747 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:29.897483    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-q7tt6"
Jul 30 17:39:30.079569 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:30.079541    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-q7tt6"]
Jul 30 17:39:30.085534 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:39:30.085472    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4338b036_d2fa_4e35_8238_a878f3223ba4.slice/crio-578b38fe167d22265321e658fef5e9fe85264e5f66036b8d2b0554cf8915b7c3 WatchSource:0}: Error finding container 578b38fe167d22265321e658fef5e9fe85264e5f66036b8d2b0554cf8915b7c3: Status 404 returned error can't find the container with id 578b38fe167d22265321e658fef5e9fe85264e5f66036b8d2b0554cf8915b7c3
Jul 30 17:39:30.098535 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:30.098501    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-9s996" event={"ID":"e7935de7-4494-46f8-ab26-5636fd38d25e","Type":"ContainerStarted","Data":"a650dab1ccbf3f1ce94de0a373669288205d4be03d7e5f971a6030840367d41c"}
Jul 30 17:39:30.100111 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:30.100047    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-q7tt6" event={"ID":"4338b036-d2fa-4e35-8238-a878f3223ba4","Type":"ContainerStarted","Data":"578b38fe167d22265321e658fef5e9fe85264e5f66036b8d2b0554cf8915b7c3"}
Jul 30 17:39:32.113742 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:32.113712    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-9s996"
Jul 30 17:39:32.114216 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:32.113781    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-9s996" event={"ID":"e7935de7-4494-46f8-ab26-5636fd38d25e","Type":"ContainerStarted","Data":"ff07ae4faa9f5d4a6cf9b9468c3c5e1b13c65ae48fb03b288b25da925648b9b6"}
Jul 30 17:39:32.130931 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:32.130867    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-9s996" podStartSLOduration=1.436743778 podStartE2EDuration="4.130845095s" totalImagesPullingTime="2.694101317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:39:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:39:32.1288436 +0000 UTC m=+746.582407593" watchObservedRunningTime="2026-07-30 17:39:32.130845095 +0000 UTC m=+746.584409086"
Jul 30 17:39:34.125388 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:34.125295    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-q7tt6" event={"ID":"4338b036-d2fa-4e35-8238-a878f3223ba4","Type":"ContainerStarted","Data":"21b503bfb0104c17fd6450441712dc00b9fc485ac1f1eb91efbfaf45d61e5ce1"}
Jul 30 17:39:34.141646 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:34.141565    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-q7tt6" podStartSLOduration=1.450250585 podStartE2EDuration="5.141546459s" totalImagesPullingTime="3.691295874s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:39:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:39:34.141130174 +0000 UTC m=+748.594694164" watchObservedRunningTime="2026-07-30 17:39:34.141546459 +0000 UTC m=+748.595110453"
Jul 30 17:39:35.789692 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:35.789657    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-q7tt6"]
Jul 30 17:39:36.134658 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:36.134559    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-q7tt6" podUID="4338b036-d2fa-4e35-8238-a878f3223ba4" containerName="authorino" containerID="cri-o://21b503bfb0104c17fd6450441712dc00b9fc485ac1f1eb91efbfaf45d61e5ce1" gracePeriod=30
Jul 30 17:39:36.382442 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:36.382413    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-q7tt6"
Jul 30 17:39:36.485256 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:36.485178    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnqzx\" (UniqueName: \"kubernetes.io/projected/4338b036-d2fa-4e35-8238-a878f3223ba4-kube-api-access-tnqzx\") pod \"4338b036-d2fa-4e35-8238-a878f3223ba4\" (UID: \"4338b036-d2fa-4e35-8238-a878f3223ba4\") "
Jul 30 17:39:36.487340 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:36.487313    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4338b036-d2fa-4e35-8238-a878f3223ba4-kube-api-access-tnqzx" (OuterVolumeSpecName: "kube-api-access-tnqzx") pod "4338b036-d2fa-4e35-8238-a878f3223ba4" (UID: "4338b036-d2fa-4e35-8238-a878f3223ba4"). InnerVolumeSpecName "kube-api-access-tnqzx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:39:36.586810 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:36.586773    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnqzx\" (UniqueName: \"kubernetes.io/projected/4338b036-d2fa-4e35-8238-a878f3223ba4-kube-api-access-tnqzx\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:39:37.141103 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:37.141071    2577 generic.go:356] "Generic (PLEG): container finished" podID="4338b036-d2fa-4e35-8238-a878f3223ba4" containerID="21b503bfb0104c17fd6450441712dc00b9fc485ac1f1eb91efbfaf45d61e5ce1" exitCode=0
Jul 30 17:39:37.141474 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:37.141126    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-q7tt6"
Jul 30 17:39:37.141474 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:37.141159    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-q7tt6" event={"ID":"4338b036-d2fa-4e35-8238-a878f3223ba4","Type":"ContainerDied","Data":"21b503bfb0104c17fd6450441712dc00b9fc485ac1f1eb91efbfaf45d61e5ce1"}
Jul 30 17:39:37.141474 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:37.141207    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-q7tt6" event={"ID":"4338b036-d2fa-4e35-8238-a878f3223ba4","Type":"ContainerDied","Data":"578b38fe167d22265321e658fef5e9fe85264e5f66036b8d2b0554cf8915b7c3"}
Jul 30 17:39:37.141474 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:37.141229    2577 scope.go:117] "RemoveContainer" containerID="21b503bfb0104c17fd6450441712dc00b9fc485ac1f1eb91efbfaf45d61e5ce1"
Jul 30 17:39:37.150082 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:37.150064    2577 scope.go:117] "RemoveContainer" containerID="21b503bfb0104c17fd6450441712dc00b9fc485ac1f1eb91efbfaf45d61e5ce1"
Jul 30 17:39:37.150310 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:39:37.150292    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21b503bfb0104c17fd6450441712dc00b9fc485ac1f1eb91efbfaf45d61e5ce1\": container with ID starting with 21b503bfb0104c17fd6450441712dc00b9fc485ac1f1eb91efbfaf45d61e5ce1 not found: ID does not exist" containerID="21b503bfb0104c17fd6450441712dc00b9fc485ac1f1eb91efbfaf45d61e5ce1"
Jul 30 17:39:37.150354 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:37.150319    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21b503bfb0104c17fd6450441712dc00b9fc485ac1f1eb91efbfaf45d61e5ce1"} err="failed to get container status \"21b503bfb0104c17fd6450441712dc00b9fc485ac1f1eb91efbfaf45d61e5ce1\": rpc error: code = NotFound desc = could not find container \"21b503bfb0104c17fd6450441712dc00b9fc485ac1f1eb91efbfaf45d61e5ce1\": container with ID starting with 21b503bfb0104c17fd6450441712dc00b9fc485ac1f1eb91efbfaf45d61e5ce1 not found: ID does not exist"
Jul 30 17:39:37.162095 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:37.162069    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-q7tt6"]
Jul 30 17:39:37.169685 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:37.169662    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-q7tt6"]
Jul 30 17:39:37.280670 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:37.280644    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:37.382485 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:37.382455    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:38.111157 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:38.111127    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4338b036-d2fa-4e35-8238-a878f3223ba4" path="/var/lib/kubelet/pods/4338b036-d2fa-4e35-8238-a878f3223ba4/volumes"
Jul 30 17:39:41.882857 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:41.882791    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:41.983879 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:41.983843    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:42.083270 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:42.083240    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:42.141289 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:42.141200    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:43.121021 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:43.120988    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-9s996"
Jul 30 17:39:44.899491 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:44.899458    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:44.999964 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:44.999930    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:46.785628 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:46.785592    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-9s996"]
Jul 30 17:39:46.786077 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:46.785791    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-9s996" podUID="e7935de7-4494-46f8-ab26-5636fd38d25e" containerName="limitador" containerID="cri-o://ff07ae4faa9f5d4a6cf9b9468c3c5e1b13c65ae48fb03b288b25da925648b9b6" gracePeriod=30
Jul 30 17:39:47.188676 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:47.188645    2577 generic.go:356] "Generic (PLEG): container finished" podID="e7935de7-4494-46f8-ab26-5636fd38d25e" containerID="ff07ae4faa9f5d4a6cf9b9468c3c5e1b13c65ae48fb03b288b25da925648b9b6" exitCode=0
Jul 30 17:39:47.188876 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:47.188715    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-9s996" event={"ID":"e7935de7-4494-46f8-ab26-5636fd38d25e","Type":"ContainerDied","Data":"ff07ae4faa9f5d4a6cf9b9468c3c5e1b13c65ae48fb03b288b25da925648b9b6"}
Jul 30 17:39:47.737298 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:47.737274    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-9s996"
Jul 30 17:39:47.890168 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:47.890076    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7935de7-4494-46f8-ab26-5636fd38d25e-config-file\") pod \"e7935de7-4494-46f8-ab26-5636fd38d25e\" (UID: \"e7935de7-4494-46f8-ab26-5636fd38d25e\") "
Jul 30 17:39:47.890527 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:47.890166    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmnwk\" (UniqueName: \"kubernetes.io/projected/e7935de7-4494-46f8-ab26-5636fd38d25e-kube-api-access-vmnwk\") pod \"e7935de7-4494-46f8-ab26-5636fd38d25e\" (UID: \"e7935de7-4494-46f8-ab26-5636fd38d25e\") "
Jul 30 17:39:47.890527 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:47.890463    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7935de7-4494-46f8-ab26-5636fd38d25e-config-file" (OuterVolumeSpecName: "config-file") pod "e7935de7-4494-46f8-ab26-5636fd38d25e" (UID: "e7935de7-4494-46f8-ab26-5636fd38d25e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 17:39:47.892288 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:47.892261    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7935de7-4494-46f8-ab26-5636fd38d25e-kube-api-access-vmnwk" (OuterVolumeSpecName: "kube-api-access-vmnwk") pod "e7935de7-4494-46f8-ab26-5636fd38d25e" (UID: "e7935de7-4494-46f8-ab26-5636fd38d25e"). InnerVolumeSpecName "kube-api-access-vmnwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:39:47.990852 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:47.990799    2577 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7935de7-4494-46f8-ab26-5636fd38d25e-config-file\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:39:47.990852 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:47.990848    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmnwk\" (UniqueName: \"kubernetes.io/projected/e7935de7-4494-46f8-ab26-5636fd38d25e-kube-api-access-vmnwk\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:39:48.195879 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:48.195792    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-9s996"
Jul 30 17:39:48.195879 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:48.195811    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-9s996" event={"ID":"e7935de7-4494-46f8-ab26-5636fd38d25e","Type":"ContainerDied","Data":"a650dab1ccbf3f1ce94de0a373669288205d4be03d7e5f971a6030840367d41c"}
Jul 30 17:39:48.196093 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:48.195891    2577 scope.go:117] "RemoveContainer" containerID="ff07ae4faa9f5d4a6cf9b9468c3c5e1b13c65ae48fb03b288b25da925648b9b6"
Jul 30 17:39:48.213892 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:48.213866    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-9s996"]
Jul 30 17:39:48.218244 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:48.218222    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-9s996"]
Jul 30 17:39:50.114466 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:50.113886    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7935de7-4494-46f8-ab26-5636fd38d25e" path="/var/lib/kubelet/pods/e7935de7-4494-46f8-ab26-5636fd38d25e/volumes"
Jul 30 17:39:50.118284 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:50.118252    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:50.206367 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:50.206335    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:50.766500 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:50.766463    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-75knz"]
Jul 30 17:39:50.766854 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:50.766834    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7935de7-4494-46f8-ab26-5636fd38d25e" containerName="limitador"
Jul 30 17:39:50.766854 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:50.766849    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7935de7-4494-46f8-ab26-5636fd38d25e" containerName="limitador"
Jul 30 17:39:50.766943 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:50.766874    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4338b036-d2fa-4e35-8238-a878f3223ba4" containerName="authorino"
Jul 30 17:39:50.766943 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:50.766879    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4338b036-d2fa-4e35-8238-a878f3223ba4" containerName="authorino"
Jul 30 17:39:50.766943 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:50.766929    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4338b036-d2fa-4e35-8238-a878f3223ba4" containerName="authorino"
Jul 30 17:39:50.766943 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:50.766941    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7935de7-4494-46f8-ab26-5636fd38d25e" containerName="limitador"
Jul 30 17:39:50.771140 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:50.771119    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-75knz"
Jul 30 17:39:50.773377 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:50.773354    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 30 17:39:50.773469 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:50.773358    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 30 17:39:50.773993 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:50.773978    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 17:39:50.774063 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:50.774049    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-q24zt\""
Jul 30 17:39:50.778368 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:50.778336    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-75knz"]
Jul 30 17:39:50.799854 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:50.799805    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:50.918895 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:50.918860    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f55376ff-9026-491a-9a91-563ec1ec77ea-data\") pod \"postgres-69fb4594cf-75knz\" (UID: \"f55376ff-9026-491a-9a91-563ec1ec77ea\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-75knz"
Jul 30 17:39:50.919069 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:50.918936    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9dvq\" (UniqueName: \"kubernetes.io/projected/f55376ff-9026-491a-9a91-563ec1ec77ea-kube-api-access-q9dvq\") pod \"postgres-69fb4594cf-75knz\" (UID: \"f55376ff-9026-491a-9a91-563ec1ec77ea\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-75knz"
Jul 30 17:39:51.019771 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:51.019684    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f55376ff-9026-491a-9a91-563ec1ec77ea-data\") pod \"postgres-69fb4594cf-75knz\" (UID: \"f55376ff-9026-491a-9a91-563ec1ec77ea\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-75knz"
Jul 30 17:39:51.019954 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:51.019916    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9dvq\" (UniqueName: \"kubernetes.io/projected/f55376ff-9026-491a-9a91-563ec1ec77ea-kube-api-access-q9dvq\") pod \"postgres-69fb4594cf-75knz\" (UID: \"f55376ff-9026-491a-9a91-563ec1ec77ea\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-75knz"
Jul 30 17:39:51.020110 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:51.020088    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f55376ff-9026-491a-9a91-563ec1ec77ea-data\") pod \"postgres-69fb4594cf-75knz\" (UID: \"f55376ff-9026-491a-9a91-563ec1ec77ea\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-75knz"
Jul 30 17:39:51.027566 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:51.027540    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9dvq\" (UniqueName: \"kubernetes.io/projected/f55376ff-9026-491a-9a91-563ec1ec77ea-kube-api-access-q9dvq\") pod \"postgres-69fb4594cf-75knz\" (UID: \"f55376ff-9026-491a-9a91-563ec1ec77ea\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-75knz"
Jul 30 17:39:51.083566 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:51.083536    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-75knz"
Jul 30 17:39:51.218468 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:51.218444    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-75knz"]
Jul 30 17:39:51.220035 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:39:51.220013    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf55376ff_9026_491a_9a91_563ec1ec77ea.slice/crio-0287eaa476464a9eb7042c3a337ac7f67048a6bebe5f38165b28356ac40a4367 WatchSource:0}: Error finding container 0287eaa476464a9eb7042c3a337ac7f67048a6bebe5f38165b28356ac40a4367: Status 404 returned error can't find the container with id 0287eaa476464a9eb7042c3a337ac7f67048a6bebe5f38165b28356ac40a4367
Jul 30 17:39:52.221910 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:52.221868    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-75knz" event={"ID":"f55376ff-9026-491a-9a91-563ec1ec77ea","Type":"ContainerStarted","Data":"0287eaa476464a9eb7042c3a337ac7f67048a6bebe5f38165b28356ac40a4367"}
Jul 30 17:39:54.547097 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:54.547052    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:54.647796 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:54.647761    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:39:56.244694 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:56.244609    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-75knz" event={"ID":"f55376ff-9026-491a-9a91-563ec1ec77ea","Type":"ContainerStarted","Data":"0b2d993bf9d9846de9afeae4a1f647f230f069f3f44c0cd60fdbc41f12cf991d"}
Jul 30 17:39:56.245075 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:56.244748    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-75knz"
Jul 30 17:39:56.266029 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:39:56.265967    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-75knz" podStartSLOduration=1.507999158 podStartE2EDuration="6.265945466s" totalImagesPullingTime="4.757946308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:39:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:39:56.263098973 +0000 UTC m=+770.716662964" watchObservedRunningTime="2026-07-30 17:39:56.265945466 +0000 UTC m=+770.719509460"
Jul 30 17:40:02.278349 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:02.278324    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-75knz"
Jul 30 17:40:02.613499 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:02.613421    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:40:02.858035 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:02.857980    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ndmnm"]
Jul 30 17:40:02.869365 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:02.869272    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ndmnm"]
Jul 30 17:40:02.869513 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:02.869390    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ndmnm"
Jul 30 17:40:02.871770 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:02.871748    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-d8xl2\""
Jul 30 17:40:02.934483 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:02.934454    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4brhn\" (UniqueName: \"kubernetes.io/projected/2fbc6029-11a9-450d-a094-ae695cdba1af-kube-api-access-4brhn\") pod \"authorino-5d4b67c446-ndmnm\" (UID: \"2fbc6029-11a9-450d-a094-ae695cdba1af\") " pod="kuadrant-system/authorino-5d4b67c446-ndmnm"
Jul 30 17:40:02.969121 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:02.969080    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:40:03.035567 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.035533    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4brhn\" (UniqueName: \"kubernetes.io/projected/2fbc6029-11a9-450d-a094-ae695cdba1af-kube-api-access-4brhn\") pod \"authorino-5d4b67c446-ndmnm\" (UID: \"2fbc6029-11a9-450d-a094-ae695cdba1af\") " pod="kuadrant-system/authorino-5d4b67c446-ndmnm"
Jul 30 17:40:03.043785 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.043747    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4brhn\" (UniqueName: \"kubernetes.io/projected/2fbc6029-11a9-450d-a094-ae695cdba1af-kube-api-access-4brhn\") pod \"authorino-5d4b67c446-ndmnm\" (UID: \"2fbc6029-11a9-450d-a094-ae695cdba1af\") " pod="kuadrant-system/authorino-5d4b67c446-ndmnm"
Jul 30 17:40:03.112625 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.112590    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ndmnm"]
Jul 30 17:40:03.112872 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.112860    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ndmnm"
Jul 30 17:40:03.139603 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.139577    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7b85f8864-ppf2z"]
Jul 30 17:40:03.155487 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.155460    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b85f8864-ppf2z"]
Jul 30 17:40:03.155636 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.155562    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b85f8864-ppf2z"
Jul 30 17:40:03.227325 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.227287    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b85f8864-ppf2z"]
Jul 30 17:40:03.227647 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:40:03.227621    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-9q549], unattached volumes=[], failed to process volumes=[kube-api-access-9q549]: context canceled" pod="kuadrant-system/authorino-7b85f8864-ppf2z" podUID="2a5c83e4-14f8-442e-8d49-441166528c1a"
Jul 30 17:40:03.238402 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.238364    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q549\" (UniqueName: \"kubernetes.io/projected/2a5c83e4-14f8-442e-8d49-441166528c1a-kube-api-access-9q549\") pod \"authorino-7b85f8864-ppf2z\" (UID: \"2a5c83e4-14f8-442e-8d49-441166528c1a\") " pod="kuadrant-system/authorino-7b85f8864-ppf2z"
Jul 30 17:40:03.256439 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.256412    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-66654fd656-x24v4"]
Jul 30 17:40:03.265370 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.265344    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66654fd656-x24v4"
Jul 30 17:40:03.267870 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.267844    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 30 17:40:03.268058 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.267879    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66654fd656-x24v4"]
Jul 30 17:40:03.270203 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.270178    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ndmnm"]
Jul 30 17:40:03.273676 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:40:03.273653    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fbc6029_11a9_450d_a094_ae695cdba1af.slice/crio-2184ff93ab3f03af53f548734d63f0a7a15b8b29f0139c09b0701f4eebe522e5 WatchSource:0}: Error finding container 2184ff93ab3f03af53f548734d63f0a7a15b8b29f0139c09b0701f4eebe522e5: Status 404 returned error can't find the container with id 2184ff93ab3f03af53f548734d63f0a7a15b8b29f0139c09b0701f4eebe522e5
Jul 30 17:40:03.277606 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.277585    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b85f8864-ppf2z"
Jul 30 17:40:03.299839 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.299801    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b85f8864-ppf2z"
Jul 30 17:40:03.339151 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.339121    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f954e949-95b1-4ed5-8244-5d421e187bf2-tls-cert\") pod \"authorino-66654fd656-x24v4\" (UID: \"f954e949-95b1-4ed5-8244-5d421e187bf2\") " pod="kuadrant-system/authorino-66654fd656-x24v4"
Jul 30 17:40:03.339291 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.339170    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q549\" (UniqueName: \"kubernetes.io/projected/2a5c83e4-14f8-442e-8d49-441166528c1a-kube-api-access-9q549\") pod \"authorino-7b85f8864-ppf2z\" (UID: \"2a5c83e4-14f8-442e-8d49-441166528c1a\") " pod="kuadrant-system/authorino-7b85f8864-ppf2z"
Jul 30 17:40:03.339291 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.339227    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx9qz\" (UniqueName: \"kubernetes.io/projected/f954e949-95b1-4ed5-8244-5d421e187bf2-kube-api-access-gx9qz\") pod \"authorino-66654fd656-x24v4\" (UID: \"f954e949-95b1-4ed5-8244-5d421e187bf2\") " pod="kuadrant-system/authorino-66654fd656-x24v4"
Jul 30 17:40:03.348274 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.348250    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q549\" (UniqueName: \"kubernetes.io/projected/2a5c83e4-14f8-442e-8d49-441166528c1a-kube-api-access-9q549\") pod \"authorino-7b85f8864-ppf2z\" (UID: \"2a5c83e4-14f8-442e-8d49-441166528c1a\") " pod="kuadrant-system/authorino-7b85f8864-ppf2z"
Jul 30 17:40:03.439649 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.439585    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9q549\" (UniqueName: \"kubernetes.io/projected/2a5c83e4-14f8-442e-8d49-441166528c1a-kube-api-access-9q549\") pod \"2a5c83e4-14f8-442e-8d49-441166528c1a\" (UID: \"2a5c83e4-14f8-442e-8d49-441166528c1a\") "
Jul 30 17:40:03.439783 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.439734    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f954e949-95b1-4ed5-8244-5d421e187bf2-tls-cert\") pod \"authorino-66654fd656-x24v4\" (UID: \"f954e949-95b1-4ed5-8244-5d421e187bf2\") " pod="kuadrant-system/authorino-66654fd656-x24v4"
Jul 30 17:40:03.439857 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.439783    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx9qz\" (UniqueName: \"kubernetes.io/projected/f954e949-95b1-4ed5-8244-5d421e187bf2-kube-api-access-gx9qz\") pod \"authorino-66654fd656-x24v4\" (UID: \"f954e949-95b1-4ed5-8244-5d421e187bf2\") " pod="kuadrant-system/authorino-66654fd656-x24v4"
Jul 30 17:40:03.441727 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.441697    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a5c83e4-14f8-442e-8d49-441166528c1a-kube-api-access-9q549" (OuterVolumeSpecName: "kube-api-access-9q549") pod "2a5c83e4-14f8-442e-8d49-441166528c1a" (UID: "2a5c83e4-14f8-442e-8d49-441166528c1a"). InnerVolumeSpecName "kube-api-access-9q549". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:40:03.442205 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.442188    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f954e949-95b1-4ed5-8244-5d421e187bf2-tls-cert\") pod \"authorino-66654fd656-x24v4\" (UID: \"f954e949-95b1-4ed5-8244-5d421e187bf2\") " pod="kuadrant-system/authorino-66654fd656-x24v4"
Jul 30 17:40:03.448029 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.448008    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx9qz\" (UniqueName: \"kubernetes.io/projected/f954e949-95b1-4ed5-8244-5d421e187bf2-kube-api-access-gx9qz\") pod \"authorino-66654fd656-x24v4\" (UID: \"f954e949-95b1-4ed5-8244-5d421e187bf2\") " pod="kuadrant-system/authorino-66654fd656-x24v4"
Jul 30 17:40:03.541141 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.541110    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9q549\" (UniqueName: \"kubernetes.io/projected/2a5c83e4-14f8-442e-8d49-441166528c1a-kube-api-access-9q549\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:40:03.590653 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.590626    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66654fd656-x24v4"
Jul 30 17:40:03.747568 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:03.747540    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66654fd656-x24v4"]
Jul 30 17:40:03.749918 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:40:03.749890    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf954e949_95b1_4ed5_8244_5d421e187bf2.slice/crio-350740899bddb845b681a7779e654f68938d4ca9df45334ee7b4c2a76409d5ca WatchSource:0}: Error finding container 350740899bddb845b681a7779e654f68938d4ca9df45334ee7b4c2a76409d5ca: Status 404 returned error can't find the container with id 350740899bddb845b681a7779e654f68938d4ca9df45334ee7b4c2a76409d5ca
Jul 30 17:40:04.283502 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:04.283475    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-ndmnm" event={"ID":"2fbc6029-11a9-450d-a094-ae695cdba1af","Type":"ContainerStarted","Data":"b18f197e2512ac5bb01cbc4528845bc8bbb6ed2b9cbf68e2583d4a1b6976039c"}
Jul 30 17:40:04.283638 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:04.283513    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-ndmnm" event={"ID":"2fbc6029-11a9-450d-a094-ae695cdba1af","Type":"ContainerStarted","Data":"2184ff93ab3f03af53f548734d63f0a7a15b8b29f0139c09b0701f4eebe522e5"}
Jul 30 17:40:04.283638 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:04.283513    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-ndmnm" podUID="2fbc6029-11a9-450d-a094-ae695cdba1af" containerName="authorino" containerID="cri-o://b18f197e2512ac5bb01cbc4528845bc8bbb6ed2b9cbf68e2583d4a1b6976039c" gracePeriod=30
Jul 30 17:40:04.285015 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:04.284993    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b85f8864-ppf2z"
Jul 30 17:40:04.285015 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:04.285000    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66654fd656-x24v4" event={"ID":"f954e949-95b1-4ed5-8244-5d421e187bf2","Type":"ContainerStarted","Data":"350740899bddb845b681a7779e654f68938d4ca9df45334ee7b4c2a76409d5ca"}
Jul 30 17:40:04.303245 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:04.303170    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-ndmnm" podStartSLOduration=1.742794181 podStartE2EDuration="2.303154193s" totalImagesPullingTime="560.360012ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:40:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:40:04.301318813 +0000 UTC m=+778.754882823" watchObservedRunningTime="2026-07-30 17:40:04.303154193 +0000 UTC m=+778.756718183"
Jul 30 17:40:04.328051 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:04.328022    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b85f8864-ppf2z"]
Jul 30 17:40:04.330933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:04.330910    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7b85f8864-ppf2z"]
Jul 30 17:40:04.559968 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:04.559902    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ndmnm"
Jul 30 17:40:04.649504 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:04.649471    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4brhn\" (UniqueName: \"kubernetes.io/projected/2fbc6029-11a9-450d-a094-ae695cdba1af-kube-api-access-4brhn\") pod \"2fbc6029-11a9-450d-a094-ae695cdba1af\" (UID: \"2fbc6029-11a9-450d-a094-ae695cdba1af\") "
Jul 30 17:40:04.651670 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:04.651643    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fbc6029-11a9-450d-a094-ae695cdba1af-kube-api-access-4brhn" (OuterVolumeSpecName: "kube-api-access-4brhn") pod "2fbc6029-11a9-450d-a094-ae695cdba1af" (UID: "2fbc6029-11a9-450d-a094-ae695cdba1af"). InnerVolumeSpecName "kube-api-access-4brhn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:40:04.750382 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:04.750350    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4brhn\" (UniqueName: \"kubernetes.io/projected/2fbc6029-11a9-450d-a094-ae695cdba1af-kube-api-access-4brhn\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:40:05.290714 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:05.290678    2577 generic.go:356] "Generic (PLEG): container finished" podID="2fbc6029-11a9-450d-a094-ae695cdba1af" containerID="b18f197e2512ac5bb01cbc4528845bc8bbb6ed2b9cbf68e2583d4a1b6976039c" exitCode=0
Jul 30 17:40:05.290899 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:05.290721    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ndmnm"
Jul 30 17:40:05.290899 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:05.290757    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-ndmnm" event={"ID":"2fbc6029-11a9-450d-a094-ae695cdba1af","Type":"ContainerDied","Data":"b18f197e2512ac5bb01cbc4528845bc8bbb6ed2b9cbf68e2583d4a1b6976039c"}
Jul 30 17:40:05.290899 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:05.290795    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-ndmnm" event={"ID":"2fbc6029-11a9-450d-a094-ae695cdba1af","Type":"ContainerDied","Data":"2184ff93ab3f03af53f548734d63f0a7a15b8b29f0139c09b0701f4eebe522e5"}
Jul 30 17:40:05.290899 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:05.290832    2577 scope.go:117] "RemoveContainer" containerID="b18f197e2512ac5bb01cbc4528845bc8bbb6ed2b9cbf68e2583d4a1b6976039c"
Jul 30 17:40:05.292723 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:05.292695    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66654fd656-x24v4" event={"ID":"f954e949-95b1-4ed5-8244-5d421e187bf2","Type":"ContainerStarted","Data":"4f79d84d972cc6d84db1aa25e6bf90492bc8c8cb82c11bd64eccd33e5edc279e"}
Jul 30 17:40:05.300299 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:05.300279    2577 scope.go:117] "RemoveContainer" containerID="b18f197e2512ac5bb01cbc4528845bc8bbb6ed2b9cbf68e2583d4a1b6976039c"
Jul 30 17:40:05.300568 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:40:05.300551    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b18f197e2512ac5bb01cbc4528845bc8bbb6ed2b9cbf68e2583d4a1b6976039c\": container with ID starting with b18f197e2512ac5bb01cbc4528845bc8bbb6ed2b9cbf68e2583d4a1b6976039c not found: ID does not exist" containerID="b18f197e2512ac5bb01cbc4528845bc8bbb6ed2b9cbf68e2583d4a1b6976039c"
Jul 30 17:40:05.300611 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:05.300578    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b18f197e2512ac5bb01cbc4528845bc8bbb6ed2b9cbf68e2583d4a1b6976039c"} err="failed to get container status \"b18f197e2512ac5bb01cbc4528845bc8bbb6ed2b9cbf68e2583d4a1b6976039c\": rpc error: code = NotFound desc = could not find container \"b18f197e2512ac5bb01cbc4528845bc8bbb6ed2b9cbf68e2583d4a1b6976039c\": container with ID starting with b18f197e2512ac5bb01cbc4528845bc8bbb6ed2b9cbf68e2583d4a1b6976039c not found: ID does not exist"
Jul 30 17:40:05.312101 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:05.312062    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-66654fd656-x24v4" podStartSLOduration=1.8324832500000001 podStartE2EDuration="2.312049268s" totalImagesPullingTime="479.566018ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:40:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:40:05.310205052 +0000 UTC m=+779.763769045" watchObservedRunningTime="2026-07-30 17:40:05.312049268 +0000 UTC m=+779.765613283"
Jul 30 17:40:05.326709 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:05.326683    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ndmnm"]
Jul 30 17:40:05.334717 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:05.334694    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ndmnm"]
Jul 30 17:40:06.116219 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:06.116183    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a5c83e4-14f8-442e-8d49-441166528c1a" path="/var/lib/kubelet/pods/2a5c83e4-14f8-442e-8d49-441166528c1a/volumes"
Jul 30 17:40:06.116421 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:06.116409    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fbc6029-11a9-450d-a094-ae695cdba1af" path="/var/lib/kubelet/pods/2fbc6029-11a9-450d-a094-ae695cdba1af/volumes"
Jul 30 17:40:09.963488 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:09.963452    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:40:10.177223 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:10.177190    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:40:10.198243 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:10.198213    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-54b68cccb5-458fd"]
Jul 30 17:40:10.198591 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:10.198579    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fbc6029-11a9-450d-a094-ae695cdba1af" containerName="authorino"
Jul 30 17:40:10.198638 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:10.198593    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fbc6029-11a9-450d-a094-ae695cdba1af" containerName="authorino"
Jul 30 17:40:10.198673 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:10.198658    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fbc6029-11a9-450d-a094-ae695cdba1af" containerName="authorino"
Jul 30 17:40:10.201922 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:10.201900    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54b68cccb5-458fd"
Jul 30 17:40:10.204593 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:10.204574    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 30 17:40:10.204722 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:10.204608    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 30 17:40:10.205316 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:10.205301    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-n5rvf\""
Jul 30 17:40:10.216650 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:10.216594    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-54b68cccb5-458fd"]
Jul 30 17:40:10.284435 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:10.284407    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:40:10.299614 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:10.299591    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/62917cd3-0cdb-4d76-a25f-5e58323568c8-webhook-cert\") pod \"maas-controller-54b68cccb5-458fd\" (UID: \"62917cd3-0cdb-4d76-a25f-5e58323568c8\") " pod="opendatahub/maas-controller-54b68cccb5-458fd"
Jul 30 17:40:10.299732 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:10.299634    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljb6x\" (UniqueName: \"kubernetes.io/projected/62917cd3-0cdb-4d76-a25f-5e58323568c8-kube-api-access-ljb6x\") pod \"maas-controller-54b68cccb5-458fd\" (UID: \"62917cd3-0cdb-4d76-a25f-5e58323568c8\") " pod="opendatahub/maas-controller-54b68cccb5-458fd"
Jul 30 17:40:10.388665 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:10.388634    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:40:10.401166 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:10.401137    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/62917cd3-0cdb-4d76-a25f-5e58323568c8-webhook-cert\") pod \"maas-controller-54b68cccb5-458fd\" (UID: \"62917cd3-0cdb-4d76-a25f-5e58323568c8\") " pod="opendatahub/maas-controller-54b68cccb5-458fd"
Jul 30 17:40:10.401329 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:10.401173    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljb6x\" (UniqueName: \"kubernetes.io/projected/62917cd3-0cdb-4d76-a25f-5e58323568c8-kube-api-access-ljb6x\") pod \"maas-controller-54b68cccb5-458fd\" (UID: \"62917cd3-0cdb-4d76-a25f-5e58323568c8\") " pod="opendatahub/maas-controller-54b68cccb5-458fd"
Jul 30 17:40:10.401329 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:40:10.401314    2577 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 30 17:40:10.401448 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:40:10.401391    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62917cd3-0cdb-4d76-a25f-5e58323568c8-webhook-cert podName:62917cd3-0cdb-4d76-a25f-5e58323568c8 nodeName:}" failed. No retries permitted until 2026-07-30 17:40:10.901367762 +0000 UTC m=+785.354931745 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/62917cd3-0cdb-4d76-a25f-5e58323568c8-webhook-cert") pod "maas-controller-54b68cccb5-458fd" (UID: "62917cd3-0cdb-4d76-a25f-5e58323568c8") : secret "maas-controller-webhook-cert" not found
Jul 30 17:40:10.414734 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:10.414707    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljb6x\" (UniqueName: \"kubernetes.io/projected/62917cd3-0cdb-4d76-a25f-5e58323568c8-kube-api-access-ljb6x\") pod \"maas-controller-54b68cccb5-458fd\" (UID: \"62917cd3-0cdb-4d76-a25f-5e58323568c8\") " pod="opendatahub/maas-controller-54b68cccb5-458fd"
Jul 30 17:40:10.452402 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:10.452372    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:40:10.905613 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:10.905577    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/62917cd3-0cdb-4d76-a25f-5e58323568c8-webhook-cert\") pod \"maas-controller-54b68cccb5-458fd\" (UID: \"62917cd3-0cdb-4d76-a25f-5e58323568c8\") " pod="opendatahub/maas-controller-54b68cccb5-458fd"
Jul 30 17:40:10.908110 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:10.908091    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/62917cd3-0cdb-4d76-a25f-5e58323568c8-webhook-cert\") pod \"maas-controller-54b68cccb5-458fd\" (UID: \"62917cd3-0cdb-4d76-a25f-5e58323568c8\") " pod="opendatahub/maas-controller-54b68cccb5-458fd"
Jul 30 17:40:11.111938 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:11.111899    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54b68cccb5-458fd"
Jul 30 17:40:11.243452 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:11.243422    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-54b68cccb5-458fd"]
Jul 30 17:40:11.245011 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:40:11.244986    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62917cd3_0cdb_4d76_a25f_5e58323568c8.slice/crio-761473f64731cae3c8acaaae6d8894f73eb16ef262610081657752840321af3d WatchSource:0}: Error finding container 761473f64731cae3c8acaaae6d8894f73eb16ef262610081657752840321af3d: Status 404 returned error can't find the container with id 761473f64731cae3c8acaaae6d8894f73eb16ef262610081657752840321af3d
Jul 30 17:40:11.320679 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:11.320644    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54b68cccb5-458fd" event={"ID":"62917cd3-0cdb-4d76-a25f-5e58323568c8","Type":"ContainerStarted","Data":"761473f64731cae3c8acaaae6d8894f73eb16ef262610081657752840321af3d"}
Jul 30 17:40:13.280963 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:13.280926    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:40:13.384198 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:13.384128    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:40:14.336856 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:14.336807    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54b68cccb5-458fd" event={"ID":"62917cd3-0cdb-4d76-a25f-5e58323568c8","Type":"ContainerStarted","Data":"5d20a797b6aab0d000ba1741b76acb78aef18994c43dd805fc00bdc464f64d63"}
Jul 30 17:40:14.337352 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:14.336936    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-54b68cccb5-458fd"
Jul 30 17:40:14.356337 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:14.356288    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-54b68cccb5-458fd" podStartSLOduration=2.167762118 podStartE2EDuration="4.356273947s" totalImagesPullingTime="2.188511829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:40:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:40:14.354220721 +0000 UTC m=+788.807784712" watchObservedRunningTime="2026-07-30 17:40:14.356273947 +0000 UTC m=+788.809837937"
Jul 30 17:40:24.966159 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:24.966128    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-54b68cccb5-458fd"]
Jul 30 17:40:24.966617 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:24.966342    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-54b68cccb5-458fd" podUID="62917cd3-0cdb-4d76-a25f-5e58323568c8" containerName="manager" containerID="cri-o://5d20a797b6aab0d000ba1741b76acb78aef18994c43dd805fc00bdc464f64d63" gracePeriod=30
Jul 30 17:40:24.968337 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:24.968316    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-54b68cccb5-458fd"
Jul 30 17:40:25.216952 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:25.216890    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54b68cccb5-458fd"
Jul 30 17:40:25.338690 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:25.338661    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljb6x\" (UniqueName: \"kubernetes.io/projected/62917cd3-0cdb-4d76-a25f-5e58323568c8-kube-api-access-ljb6x\") pod \"62917cd3-0cdb-4d76-a25f-5e58323568c8\" (UID: \"62917cd3-0cdb-4d76-a25f-5e58323568c8\") "
Jul 30 17:40:25.338908 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:25.338708    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/62917cd3-0cdb-4d76-a25f-5e58323568c8-webhook-cert\") pod \"62917cd3-0cdb-4d76-a25f-5e58323568c8\" (UID: \"62917cd3-0cdb-4d76-a25f-5e58323568c8\") "
Jul 30 17:40:25.341083 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:25.341056    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62917cd3-0cdb-4d76-a25f-5e58323568c8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "62917cd3-0cdb-4d76-a25f-5e58323568c8" (UID: "62917cd3-0cdb-4d76-a25f-5e58323568c8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:40:25.341193 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:25.341127    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62917cd3-0cdb-4d76-a25f-5e58323568c8-kube-api-access-ljb6x" (OuterVolumeSpecName: "kube-api-access-ljb6x") pod "62917cd3-0cdb-4d76-a25f-5e58323568c8" (UID: "62917cd3-0cdb-4d76-a25f-5e58323568c8"). InnerVolumeSpecName "kube-api-access-ljb6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:40:25.389645 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:25.389610    2577 generic.go:356] "Generic (PLEG): container finished" podID="62917cd3-0cdb-4d76-a25f-5e58323568c8" containerID="5d20a797b6aab0d000ba1741b76acb78aef18994c43dd805fc00bdc464f64d63" exitCode=0
Jul 30 17:40:25.389795 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:25.389658    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54b68cccb5-458fd"
Jul 30 17:40:25.389795 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:25.389696    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54b68cccb5-458fd" event={"ID":"62917cd3-0cdb-4d76-a25f-5e58323568c8","Type":"ContainerDied","Data":"5d20a797b6aab0d000ba1741b76acb78aef18994c43dd805fc00bdc464f64d63"}
Jul 30 17:40:25.389795 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:25.389750    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54b68cccb5-458fd" event={"ID":"62917cd3-0cdb-4d76-a25f-5e58323568c8","Type":"ContainerDied","Data":"761473f64731cae3c8acaaae6d8894f73eb16ef262610081657752840321af3d"}
Jul 30 17:40:25.389795 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:25.389773    2577 scope.go:117] "RemoveContainer" containerID="5d20a797b6aab0d000ba1741b76acb78aef18994c43dd805fc00bdc464f64d63"
Jul 30 17:40:25.398553 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:25.398535    2577 scope.go:117] "RemoveContainer" containerID="5d20a797b6aab0d000ba1741b76acb78aef18994c43dd805fc00bdc464f64d63"
Jul 30 17:40:25.398911 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:40:25.398792    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d20a797b6aab0d000ba1741b76acb78aef18994c43dd805fc00bdc464f64d63\": container with ID starting with 5d20a797b6aab0d000ba1741b76acb78aef18994c43dd805fc00bdc464f64d63 not found: ID does not exist" containerID="5d20a797b6aab0d000ba1741b76acb78aef18994c43dd805fc00bdc464f64d63"
Jul 30 17:40:25.398911 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:25.398840    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d20a797b6aab0d000ba1741b76acb78aef18994c43dd805fc00bdc464f64d63"} err="failed to get container status \"5d20a797b6aab0d000ba1741b76acb78aef18994c43dd805fc00bdc464f64d63\": rpc error: code = NotFound desc = could not find container \"5d20a797b6aab0d000ba1741b76acb78aef18994c43dd805fc00bdc464f64d63\": container with ID starting with 5d20a797b6aab0d000ba1741b76acb78aef18994c43dd805fc00bdc464f64d63 not found: ID does not exist"
Jul 30 17:40:25.412593 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:25.412567    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-54b68cccb5-458fd"]
Jul 30 17:40:25.416724 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:25.416701    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-54b68cccb5-458fd"]
Jul 30 17:40:25.439604 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:25.439578    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljb6x\" (UniqueName: \"kubernetes.io/projected/62917cd3-0cdb-4d76-a25f-5e58323568c8-kube-api-access-ljb6x\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:40:25.439604 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:25.439605    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/62917cd3-0cdb-4d76-a25f-5e58323568c8-webhook-cert\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:40:26.111304 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:26.111270    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62917cd3-0cdb-4d76-a25f-5e58323568c8" path="/var/lib/kubelet/pods/62917cd3-0cdb-4d76-a25f-5e58323568c8/volumes"
Jul 30 17:40:31.847045 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:31.846999    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:40:31.948841 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:31.948792    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:40:44.873313 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:44.873280    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:40:44.998929 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:44.998871    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:40:46.516779 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:46.516748    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:40:46.562387 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:46.562353    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-ff54b6d89-67f82"]
Jul 30 17:40:46.562741 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:46.562729    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62917cd3-0cdb-4d76-a25f-5e58323568c8" containerName="manager"
Jul 30 17:40:46.562791 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:46.562742    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="62917cd3-0cdb-4d76-a25f-5e58323568c8" containerName="manager"
Jul 30 17:40:46.562845 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:46.562803    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="62917cd3-0cdb-4d76-a25f-5e58323568c8" containerName="manager"
Jul 30 17:40:46.569400 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:46.569376    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-ff54b6d89-67f82"
Jul 30 17:40:46.571526 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:46.571508    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 30 17:40:46.571597 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:46.571549    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9z8wr\""
Jul 30 17:40:46.574794 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:46.574770    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-ff54b6d89-67f82"]
Jul 30 17:40:46.630138 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:46.630099    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdae6d27-dd7b-4f7c-b92c-50282580fb2d-maas-api-tls\") pod \"maas-api-ff54b6d89-67f82\" (UID: \"cdae6d27-dd7b-4f7c-b92c-50282580fb2d\") " pod="odh-ai-gateway-infra/maas-api-ff54b6d89-67f82"
Jul 30 17:40:46.630321 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:46.630175    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t68q\" (UniqueName: \"kubernetes.io/projected/cdae6d27-dd7b-4f7c-b92c-50282580fb2d-kube-api-access-4t68q\") pod \"maas-api-ff54b6d89-67f82\" (UID: \"cdae6d27-dd7b-4f7c-b92c-50282580fb2d\") " pod="odh-ai-gateway-infra/maas-api-ff54b6d89-67f82"
Jul 30 17:40:46.630321 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:46.630208    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdae6d27-dd7b-4f7c-b92c-50282580fb2d-openshift-service-ca\") pod \"maas-api-ff54b6d89-67f82\" (UID: \"cdae6d27-dd7b-4f7c-b92c-50282580fb2d\") " pod="odh-ai-gateway-infra/maas-api-ff54b6d89-67f82"
Jul 30 17:40:46.730634 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:46.730599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t68q\" (UniqueName: \"kubernetes.io/projected/cdae6d27-dd7b-4f7c-b92c-50282580fb2d-kube-api-access-4t68q\") pod \"maas-api-ff54b6d89-67f82\" (UID: \"cdae6d27-dd7b-4f7c-b92c-50282580fb2d\") " pod="odh-ai-gateway-infra/maas-api-ff54b6d89-67f82"
Jul 30 17:40:46.730634 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:46.730647    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdae6d27-dd7b-4f7c-b92c-50282580fb2d-openshift-service-ca\") pod \"maas-api-ff54b6d89-67f82\" (UID: \"cdae6d27-dd7b-4f7c-b92c-50282580fb2d\") " pod="odh-ai-gateway-infra/maas-api-ff54b6d89-67f82"
Jul 30 17:40:46.730873 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:46.730694    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdae6d27-dd7b-4f7c-b92c-50282580fb2d-maas-api-tls\") pod \"maas-api-ff54b6d89-67f82\" (UID: \"cdae6d27-dd7b-4f7c-b92c-50282580fb2d\") " pod="odh-ai-gateway-infra/maas-api-ff54b6d89-67f82"
Jul 30 17:40:46.730873 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:40:46.730850    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 30 17:40:46.730946 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:40:46.730933    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdae6d27-dd7b-4f7c-b92c-50282580fb2d-maas-api-tls podName:cdae6d27-dd7b-4f7c-b92c-50282580fb2d nodeName:}" failed. No retries permitted until 2026-07-30 17:40:47.230909683 +0000 UTC m=+821.684473654 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cdae6d27-dd7b-4f7c-b92c-50282580fb2d-maas-api-tls") pod "maas-api-ff54b6d89-67f82" (UID: "cdae6d27-dd7b-4f7c-b92c-50282580fb2d") : secret "maas-api-serving-cert" not found
Jul 30 17:40:46.731419 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:46.731401    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdae6d27-dd7b-4f7c-b92c-50282580fb2d-openshift-service-ca\") pod \"maas-api-ff54b6d89-67f82\" (UID: \"cdae6d27-dd7b-4f7c-b92c-50282580fb2d\") " pod="odh-ai-gateway-infra/maas-api-ff54b6d89-67f82"
Jul 30 17:40:46.739358 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:46.739325    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t68q\" (UniqueName: \"kubernetes.io/projected/cdae6d27-dd7b-4f7c-b92c-50282580fb2d-kube-api-access-4t68q\") pod \"maas-api-ff54b6d89-67f82\" (UID: \"cdae6d27-dd7b-4f7c-b92c-50282580fb2d\") " pod="odh-ai-gateway-infra/maas-api-ff54b6d89-67f82"
Jul 30 17:40:46.970695 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:46.970615    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:40:47.072150 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:47.072112    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:40:47.100117 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:47.100088    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-t25kk"]
Jul 30 17:40:47.103708 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:47.103686    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-t25kk"
Jul 30 17:40:47.105717 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:47.105697    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 30 17:40:47.113486 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:47.113467    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-t25kk"]
Jul 30 17:40:47.134226 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:47.134199    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftsz9\" (UniqueName: \"kubernetes.io/projected/335be1a6-121e-41a2-b390-180e8c87ccdf-kube-api-access-ftsz9\") pod \"payload-processing-5cc6cd444b-t25kk\" (UID: \"335be1a6-121e-41a2-b390-180e8c87ccdf\") " pod="openshift-ingress/payload-processing-5cc6cd444b-t25kk"
Jul 30 17:40:47.134343 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:47.134245    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/335be1a6-121e-41a2-b390-180e8c87ccdf-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-t25kk\" (UID: \"335be1a6-121e-41a2-b390-180e8c87ccdf\") " pod="openshift-ingress/payload-processing-5cc6cd444b-t25kk"
Jul 30 17:40:47.168845 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:47.168793    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:40:47.235714 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:47.235631    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdae6d27-dd7b-4f7c-b92c-50282580fb2d-maas-api-tls\") pod \"maas-api-ff54b6d89-67f82\" (UID: \"cdae6d27-dd7b-4f7c-b92c-50282580fb2d\") " pod="odh-ai-gateway-infra/maas-api-ff54b6d89-67f82"
Jul 30 17:40:47.235714 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:47.235678    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftsz9\" (UniqueName: \"kubernetes.io/projected/335be1a6-121e-41a2-b390-180e8c87ccdf-kube-api-access-ftsz9\") pod \"payload-processing-5cc6cd444b-t25kk\" (UID: \"335be1a6-121e-41a2-b390-180e8c87ccdf\") " pod="openshift-ingress/payload-processing-5cc6cd444b-t25kk"
Jul 30 17:40:47.235983 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:47.235878    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/335be1a6-121e-41a2-b390-180e8c87ccdf-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-t25kk\" (UID: \"335be1a6-121e-41a2-b390-180e8c87ccdf\") " pod="openshift-ingress/payload-processing-5cc6cd444b-t25kk"
Jul 30 17:40:47.236569 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:47.236551    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/335be1a6-121e-41a2-b390-180e8c87ccdf-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-t25kk\" (UID: \"335be1a6-121e-41a2-b390-180e8c87ccdf\") " pod="openshift-ingress/payload-processing-5cc6cd444b-t25kk"
Jul 30 17:40:47.238546 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:47.238517    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdae6d27-dd7b-4f7c-b92c-50282580fb2d-maas-api-tls\") pod \"maas-api-ff54b6d89-67f82\" (UID: \"cdae6d27-dd7b-4f7c-b92c-50282580fb2d\") " pod="odh-ai-gateway-infra/maas-api-ff54b6d89-67f82"
Jul 30 17:40:47.250845 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:47.244661    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftsz9\" (UniqueName: \"kubernetes.io/projected/335be1a6-121e-41a2-b390-180e8c87ccdf-kube-api-access-ftsz9\") pod \"payload-processing-5cc6cd444b-t25kk\" (UID: \"335be1a6-121e-41a2-b390-180e8c87ccdf\") " pod="openshift-ingress/payload-processing-5cc6cd444b-t25kk"
Jul 30 17:40:47.415032 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:47.414973    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-t25kk"
Jul 30 17:40:47.481137 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:47.481092    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-ff54b6d89-67f82"
Jul 30 17:40:47.555852 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:47.555738    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-t25kk"]
Jul 30 17:40:47.559801 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:40:47.559775    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod335be1a6_121e_41a2_b390_180e8c87ccdf.slice/crio-82b141f8303a6e815112e98a0e43f361cf7d0a8a02548c12cd188d68f530d656 WatchSource:0}: Error finding container 82b141f8303a6e815112e98a0e43f361cf7d0a8a02548c12cd188d68f530d656: Status 404 returned error can't find the container with id 82b141f8303a6e815112e98a0e43f361cf7d0a8a02548c12cd188d68f530d656
Jul 30 17:40:47.628751 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:47.628546    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-ff54b6d89-67f82"]
Jul 30 17:40:47.630809 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:40:47.630782    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdae6d27_dd7b_4f7c_b92c_50282580fb2d.slice/crio-c897c634280b11b45b7e66cb3491e46772681b4498777fcdc122e572a74b001d WatchSource:0}: Error finding container c897c634280b11b45b7e66cb3491e46772681b4498777fcdc122e572a74b001d: Status 404 returned error can't find the container with id c897c634280b11b45b7e66cb3491e46772681b4498777fcdc122e572a74b001d
Jul 30 17:40:48.507177 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:48.507092    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-ff54b6d89-67f82" event={"ID":"cdae6d27-dd7b-4f7c-b92c-50282580fb2d","Type":"ContainerStarted","Data":"c897c634280b11b45b7e66cb3491e46772681b4498777fcdc122e572a74b001d"}
Jul 30 17:40:48.508602 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:48.508568    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-t25kk" event={"ID":"335be1a6-121e-41a2-b390-180e8c87ccdf","Type":"ContainerStarted","Data":"82b141f8303a6e815112e98a0e43f361cf7d0a8a02548c12cd188d68f530d656"}
Jul 30 17:40:50.520883 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:50.520843    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-t25kk" event={"ID":"335be1a6-121e-41a2-b390-180e8c87ccdf","Type":"ContainerStarted","Data":"e86817bda2ee0251525bba854810e505de976341e7ff1508e81eea3a5fd95be8"}
Jul 30 17:40:50.521335 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:50.521054    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-t25kk"
Jul 30 17:40:50.522390 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:50.522366    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-ff54b6d89-67f82" event={"ID":"cdae6d27-dd7b-4f7c-b92c-50282580fb2d","Type":"ContainerStarted","Data":"d1c5b1894656d639f31e7eef8f93c9c20f3c38c49778078baaf08794bef3a505"}
Jul 30 17:40:50.522475 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:50.522432    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-ff54b6d89-67f82"
Jul 30 17:40:50.537278 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:50.537233    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-t25kk" podStartSLOduration=1.4691816 podStartE2EDuration="3.537219962s" totalImagesPullingTime="2.068038362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:40:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:40:50.536472249 +0000 UTC m=+824.990036239" watchObservedRunningTime="2026-07-30 17:40:50.537219962 +0000 UTC m=+824.990783953"
Jul 30 17:40:50.553313 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:50.553267    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-ff54b6d89-67f82" podStartSLOduration=2.5626389290000002 podStartE2EDuration="4.553251833s" totalImagesPullingTime="1.990612904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:40:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:40:50.552253219 +0000 UTC m=+825.005817211" watchObservedRunningTime="2026-07-30 17:40:50.553251833 +0000 UTC m=+825.006815824"
Jul 30 17:40:56.534482 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:56.534449    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-ff54b6d89-67f82"
Jul 30 17:40:57.565673 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:57.565640    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:40:57.674933 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:40:57.674903    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:41:01.530212 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:01.530184    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-t25kk"
Jul 30 17:41:04.323587 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.323548    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"]
Jul 30 17:41:04.327759 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.327735    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:04.331123 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.331098    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-lr279\""
Jul 30 17:41:04.331123 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.331109    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 30 17:41:04.331316 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.331134    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 30 17:41:04.331316 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.331109    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 30 17:41:04.339417 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.339396    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"]
Jul 30 17:41:04.361643 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.361621    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:41:04.396004 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.395972    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17ac034f-cbc3-49de-98c1-0c0bcf06d098-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj\" (UID: \"17ac034f-cbc3-49de-98c1-0c0bcf06d098\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:04.396140 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.396010    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmrtj\" (UniqueName: \"kubernetes.io/projected/17ac034f-cbc3-49de-98c1-0c0bcf06d098-kube-api-access-jmrtj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj\" (UID: \"17ac034f-cbc3-49de-98c1-0c0bcf06d098\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:04.396140 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.396042    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17ac034f-cbc3-49de-98c1-0c0bcf06d098-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj\" (UID: \"17ac034f-cbc3-49de-98c1-0c0bcf06d098\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:04.396140 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.396126    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17ac034f-cbc3-49de-98c1-0c0bcf06d098-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj\" (UID: \"17ac034f-cbc3-49de-98c1-0c0bcf06d098\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:04.396279 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.396162    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17ac034f-cbc3-49de-98c1-0c0bcf06d098-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj\" (UID: \"17ac034f-cbc3-49de-98c1-0c0bcf06d098\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:04.396279 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.396203    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17ac034f-cbc3-49de-98c1-0c0bcf06d098-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj\" (UID: \"17ac034f-cbc3-49de-98c1-0c0bcf06d098\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:04.496800 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.496767    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17ac034f-cbc3-49de-98c1-0c0bcf06d098-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj\" (UID: \"17ac034f-cbc3-49de-98c1-0c0bcf06d098\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:04.496993 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.496862    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17ac034f-cbc3-49de-98c1-0c0bcf06d098-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj\" (UID: \"17ac034f-cbc3-49de-98c1-0c0bcf06d098\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:04.496993 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.496900    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17ac034f-cbc3-49de-98c1-0c0bcf06d098-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj\" (UID: \"17ac034f-cbc3-49de-98c1-0c0bcf06d098\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:04.496993 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.496943    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17ac034f-cbc3-49de-98c1-0c0bcf06d098-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj\" (UID: \"17ac034f-cbc3-49de-98c1-0c0bcf06d098\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:04.497108 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.497024    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17ac034f-cbc3-49de-98c1-0c0bcf06d098-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj\" (UID: \"17ac034f-cbc3-49de-98c1-0c0bcf06d098\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:04.497108 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.497065    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmrtj\" (UniqueName: \"kubernetes.io/projected/17ac034f-cbc3-49de-98c1-0c0bcf06d098-kube-api-access-jmrtj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj\" (UID: \"17ac034f-cbc3-49de-98c1-0c0bcf06d098\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:04.497248 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.497225    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17ac034f-cbc3-49de-98c1-0c0bcf06d098-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj\" (UID: \"17ac034f-cbc3-49de-98c1-0c0bcf06d098\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:04.497310 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.497274    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17ac034f-cbc3-49de-98c1-0c0bcf06d098-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj\" (UID: \"17ac034f-cbc3-49de-98c1-0c0bcf06d098\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:04.497376 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.497317    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17ac034f-cbc3-49de-98c1-0c0bcf06d098-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj\" (UID: \"17ac034f-cbc3-49de-98c1-0c0bcf06d098\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:04.499317 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.499295    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17ac034f-cbc3-49de-98c1-0c0bcf06d098-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj\" (UID: \"17ac034f-cbc3-49de-98c1-0c0bcf06d098\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:04.499608 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.499589    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17ac034f-cbc3-49de-98c1-0c0bcf06d098-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj\" (UID: \"17ac034f-cbc3-49de-98c1-0c0bcf06d098\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:04.505336 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.505316    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmrtj\" (UniqueName: \"kubernetes.io/projected/17ac034f-cbc3-49de-98c1-0c0bcf06d098-kube-api-access-jmrtj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj\" (UID: \"17ac034f-cbc3-49de-98c1-0c0bcf06d098\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:04.639495 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.639405    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:04.778807 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:04.778741    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"]
Jul 30 17:41:04.781541 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:41:04.781514    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17ac034f_cbc3_49de_98c1_0c0bcf06d098.slice/crio-3eab6450574674066973620bc0ca525335f57bf43fde25abc44a34bcc92324fc WatchSource:0}: Error finding container 3eab6450574674066973620bc0ca525335f57bf43fde25abc44a34bcc92324fc: Status 404 returned error can't find the container with id 3eab6450574674066973620bc0ca525335f57bf43fde25abc44a34bcc92324fc
Jul 30 17:41:05.599068 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:05.599032    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj" event={"ID":"17ac034f-cbc3-49de-98c1-0c0bcf06d098","Type":"ContainerStarted","Data":"3eab6450574674066973620bc0ca525335f57bf43fde25abc44a34bcc92324fc"}
Jul 30 17:41:06.605326 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:06.605289    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj" event={"ID":"17ac034f-cbc3-49de-98c1-0c0bcf06d098","Type":"ContainerStarted","Data":"3e90f2cfe6bd74dab11d4cae4fc79d9a0ab87d3f54e44528c59d9de8276ed54a"}
Jul 30 17:41:06.625835 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:06.625752    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj" podStartSLOduration=1.292458873 podStartE2EDuration="2.625732916s" totalImagesPullingTime="1.333274043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:41:06.622197109 +0000 UTC m=+841.075761100" watchObservedRunningTime="2026-07-30 17:41:06.625732916 +0000 UTC m=+841.079296908"
Jul 30 17:41:13.253126 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:13.253065    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:41:14.640086 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:14.640043    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:14.640575 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:14.640102    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:14.653426 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:14.653394    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:15.661811 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:15.661771    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj"
Jul 30 17:41:21.145672 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:21.145634    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:41:30.414958 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.414918    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"]
Jul 30 17:41:30.418599 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.418570    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:30.420860 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.420837    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 30 17:41:30.427869 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.427844    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"]
Jul 30 17:41:30.437939 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.437911    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/887b8ea5-c7a7-42c2-8d46-b4df91b4b477-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22\" (UID: \"887b8ea5-c7a7-42c2-8d46-b4df91b4b477\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:30.438097 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.437978    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/887b8ea5-c7a7-42c2-8d46-b4df91b4b477-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22\" (UID: \"887b8ea5-c7a7-42c2-8d46-b4df91b4b477\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:30.438097 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.438013    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/887b8ea5-c7a7-42c2-8d46-b4df91b4b477-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22\" (UID: \"887b8ea5-c7a7-42c2-8d46-b4df91b4b477\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:30.438225 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.438131    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/887b8ea5-c7a7-42c2-8d46-b4df91b4b477-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22\" (UID: \"887b8ea5-c7a7-42c2-8d46-b4df91b4b477\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:30.438225 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.438175    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/887b8ea5-c7a7-42c2-8d46-b4df91b4b477-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22\" (UID: \"887b8ea5-c7a7-42c2-8d46-b4df91b4b477\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:30.438225 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.438202    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbbsq\" (UniqueName: \"kubernetes.io/projected/887b8ea5-c7a7-42c2-8d46-b4df91b4b477-kube-api-access-dbbsq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22\" (UID: \"887b8ea5-c7a7-42c2-8d46-b4df91b4b477\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:30.447869 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.447809    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:41:30.539009 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.538973    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/887b8ea5-c7a7-42c2-8d46-b4df91b4b477-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22\" (UID: \"887b8ea5-c7a7-42c2-8d46-b4df91b4b477\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:30.539208 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.539036    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/887b8ea5-c7a7-42c2-8d46-b4df91b4b477-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22\" (UID: \"887b8ea5-c7a7-42c2-8d46-b4df91b4b477\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:30.539208 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.539059    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/887b8ea5-c7a7-42c2-8d46-b4df91b4b477-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22\" (UID: \"887b8ea5-c7a7-42c2-8d46-b4df91b4b477\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:30.539208 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.539108    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/887b8ea5-c7a7-42c2-8d46-b4df91b4b477-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22\" (UID: \"887b8ea5-c7a7-42c2-8d46-b4df91b4b477\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:30.539208 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.539132    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/887b8ea5-c7a7-42c2-8d46-b4df91b4b477-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22\" (UID: \"887b8ea5-c7a7-42c2-8d46-b4df91b4b477\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:30.539208 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.539151    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbbsq\" (UniqueName: \"kubernetes.io/projected/887b8ea5-c7a7-42c2-8d46-b4df91b4b477-kube-api-access-dbbsq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22\" (UID: \"887b8ea5-c7a7-42c2-8d46-b4df91b4b477\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:30.539495 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.539411    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/887b8ea5-c7a7-42c2-8d46-b4df91b4b477-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22\" (UID: \"887b8ea5-c7a7-42c2-8d46-b4df91b4b477\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:30.539495 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.539459    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/887b8ea5-c7a7-42c2-8d46-b4df91b4b477-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22\" (UID: \"887b8ea5-c7a7-42c2-8d46-b4df91b4b477\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:30.539607 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.539543    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/887b8ea5-c7a7-42c2-8d46-b4df91b4b477-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22\" (UID: \"887b8ea5-c7a7-42c2-8d46-b4df91b4b477\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:30.541526 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.541503    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/887b8ea5-c7a7-42c2-8d46-b4df91b4b477-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22\" (UID: \"887b8ea5-c7a7-42c2-8d46-b4df91b4b477\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:30.541784 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.541761    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/887b8ea5-c7a7-42c2-8d46-b4df91b4b477-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22\" (UID: \"887b8ea5-c7a7-42c2-8d46-b4df91b4b477\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:30.547333 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.547315    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbbsq\" (UniqueName: \"kubernetes.io/projected/887b8ea5-c7a7-42c2-8d46-b4df91b4b477-kube-api-access-dbbsq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22\" (UID: \"887b8ea5-c7a7-42c2-8d46-b4df91b4b477\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:30.730697 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.730593    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:30.868312 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:30.868286    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"]
Jul 30 17:41:30.871637 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:41:30.871603    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod887b8ea5_c7a7_42c2_8d46_b4df91b4b477.slice/crio-5e30508f4dd4bcc35e989af3861f9a5e72b5b69e34e7a6e2f18b7ec58eea3c17 WatchSource:0}: Error finding container 5e30508f4dd4bcc35e989af3861f9a5e72b5b69e34e7a6e2f18b7ec58eea3c17: Status 404 returned error can't find the container with id 5e30508f4dd4bcc35e989af3861f9a5e72b5b69e34e7a6e2f18b7ec58eea3c17
Jul 30 17:41:31.736748 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:31.736707    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22" event={"ID":"887b8ea5-c7a7-42c2-8d46-b4df91b4b477","Type":"ContainerStarted","Data":"a804199d682281416014ac39da3dbb582764795fbe537eb3a64382891f76d14d"}
Jul 30 17:41:31.737190 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:31.736765    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22" event={"ID":"887b8ea5-c7a7-42c2-8d46-b4df91b4b477","Type":"ContainerStarted","Data":"5e30508f4dd4bcc35e989af3861f9a5e72b5b69e34e7a6e2f18b7ec58eea3c17"}
Jul 30 17:41:31.760731 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:31.760665    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22" podStartSLOduration=1.5601780440000002 podStartE2EDuration="1.760642413s" totalImagesPullingTime="200.464369ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:41:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:41:31.758582166 +0000 UTC m=+866.212146158" watchObservedRunningTime="2026-07-30 17:41:31.760642413 +0000 UTC m=+866.214206405"
Jul 30 17:41:36.848056 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:36.848026    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:41:38.541929 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:38.541897    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:41:38.638733 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:38.638696    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:41:40.731354 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:40.731311    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:40.731859 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:40.731405    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:40.744554 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:40.744520    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:40.795649 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:40.795623    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22"
Jul 30 17:41:50.429344 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:41:50.429304    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:43:21.799232 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:21.799191    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66654fd656-x24v4"]
Jul 30 17:43:21.799846 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:21.799439    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-66654fd656-x24v4" podUID="f954e949-95b1-4ed5-8244-5d421e187bf2" containerName="authorino" containerID="cri-o://4f79d84d972cc6d84db1aa25e6bf90492bc8c8cb82c11bd64eccd33e5edc279e" gracePeriod=30
Jul 30 17:43:22.056210 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:22.056134    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66654fd656-x24v4"
Jul 30 17:43:22.179493 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:22.179460    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gx9qz\" (UniqueName: \"kubernetes.io/projected/f954e949-95b1-4ed5-8244-5d421e187bf2-kube-api-access-gx9qz\") pod \"f954e949-95b1-4ed5-8244-5d421e187bf2\" (UID: \"f954e949-95b1-4ed5-8244-5d421e187bf2\") "
Jul 30 17:43:22.179657 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:22.179539    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f954e949-95b1-4ed5-8244-5d421e187bf2-tls-cert\") pod \"f954e949-95b1-4ed5-8244-5d421e187bf2\" (UID: \"f954e949-95b1-4ed5-8244-5d421e187bf2\") "
Jul 30 17:43:22.182296 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:22.182270    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f954e949-95b1-4ed5-8244-5d421e187bf2-kube-api-access-gx9qz" (OuterVolumeSpecName: "kube-api-access-gx9qz") pod "f954e949-95b1-4ed5-8244-5d421e187bf2" (UID: "f954e949-95b1-4ed5-8244-5d421e187bf2"). InnerVolumeSpecName "kube-api-access-gx9qz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:43:22.190857 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:22.190831    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f954e949-95b1-4ed5-8244-5d421e187bf2-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "f954e949-95b1-4ed5-8244-5d421e187bf2" (UID: "f954e949-95b1-4ed5-8244-5d421e187bf2"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:43:22.280503 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:22.280472    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gx9qz\" (UniqueName: \"kubernetes.io/projected/f954e949-95b1-4ed5-8244-5d421e187bf2-kube-api-access-gx9qz\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:43:22.280503 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:22.280502    2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f954e949-95b1-4ed5-8244-5d421e187bf2-tls-cert\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:43:22.293501 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:22.293472    2577 generic.go:356] "Generic (PLEG): container finished" podID="f954e949-95b1-4ed5-8244-5d421e187bf2" containerID="4f79d84d972cc6d84db1aa25e6bf90492bc8c8cb82c11bd64eccd33e5edc279e" exitCode=0
Jul 30 17:43:22.293649 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:22.293511    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66654fd656-x24v4" event={"ID":"f954e949-95b1-4ed5-8244-5d421e187bf2","Type":"ContainerDied","Data":"4f79d84d972cc6d84db1aa25e6bf90492bc8c8cb82c11bd64eccd33e5edc279e"}
Jul 30 17:43:22.293649 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:22.293540    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66654fd656-x24v4"
Jul 30 17:43:22.293649 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:22.293554    2577 scope.go:117] "RemoveContainer" containerID="4f79d84d972cc6d84db1aa25e6bf90492bc8c8cb82c11bd64eccd33e5edc279e"
Jul 30 17:43:22.293649 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:22.293544    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66654fd656-x24v4" event={"ID":"f954e949-95b1-4ed5-8244-5d421e187bf2","Type":"ContainerDied","Data":"350740899bddb845b681a7779e654f68938d4ca9df45334ee7b4c2a76409d5ca"}
Jul 30 17:43:22.305486 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:22.305460    2577 scope.go:117] "RemoveContainer" containerID="4f79d84d972cc6d84db1aa25e6bf90492bc8c8cb82c11bd64eccd33e5edc279e"
Jul 30 17:43:22.306081 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:43:22.306060    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f79d84d972cc6d84db1aa25e6bf90492bc8c8cb82c11bd64eccd33e5edc279e\": container with ID starting with 4f79d84d972cc6d84db1aa25e6bf90492bc8c8cb82c11bd64eccd33e5edc279e not found: ID does not exist" containerID="4f79d84d972cc6d84db1aa25e6bf90492bc8c8cb82c11bd64eccd33e5edc279e"
Jul 30 17:43:22.306176 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:22.306090    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f79d84d972cc6d84db1aa25e6bf90492bc8c8cb82c11bd64eccd33e5edc279e"} err="failed to get container status \"4f79d84d972cc6d84db1aa25e6bf90492bc8c8cb82c11bd64eccd33e5edc279e\": rpc error: code = NotFound desc = could not find container \"4f79d84d972cc6d84db1aa25e6bf90492bc8c8cb82c11bd64eccd33e5edc279e\": container with ID starting with 4f79d84d972cc6d84db1aa25e6bf90492bc8c8cb82c11bd64eccd33e5edc279e not found: ID does not exist"
Jul 30 17:43:22.330323 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:22.330247    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66654fd656-x24v4"]
Jul 30 17:43:22.336729 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:22.336695    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-66654fd656-x24v4"]
Jul 30 17:43:22.413730 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:22.413698    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:43:22.485240 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:22.485211    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:43:23.564400 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:23.564340    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:43:23.691932 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:23.691899    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:43:24.111174 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:24.111137    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f954e949-95b1-4ed5-8244-5d421e187bf2" path="/var/lib/kubelet/pods/f954e949-95b1-4ed5-8244-5d421e187bf2/volumes"
Jul 30 17:43:42.627675 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:43:42.627633    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:44:19.289515 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:44:19.289478    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:44:43.509161 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:44:43.509121    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:45:00.146896 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:00.146783    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757225-7lrcp"]
Jul 30 17:45:00.154071 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:00.147789    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f954e949-95b1-4ed5-8244-5d421e187bf2" containerName="authorino"
Jul 30 17:45:00.154071 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:00.147846    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f954e949-95b1-4ed5-8244-5d421e187bf2" containerName="authorino"
Jul 30 17:45:00.154071 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:00.148231    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f954e949-95b1-4ed5-8244-5d421e187bf2" containerName="authorino"
Jul 30 17:45:00.154071 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:00.152859    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757225-7lrcp"
Jul 30 17:45:00.155986 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:00.155960    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qw74k\""
Jul 30 17:45:00.157719 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:00.157684    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757225-7lrcp"]
Jul 30 17:45:00.167196 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:00.167174    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757225-7lrcp"
Jul 30 17:45:00.297801 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:00.297777    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757225-7lrcp"]
Jul 30 17:45:00.300541 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:45:00.300498    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod940576c4_8056_4abe_99d1_253cefffb3c3.slice/crio-72eaee0b882cee5fa2e8423c2dff8edfe665c97d85ce48626c60931f27a9214c WatchSource:0}: Error finding container 72eaee0b882cee5fa2e8423c2dff8edfe665c97d85ce48626c60931f27a9214c: Status 404 returned error can't find the container with id 72eaee0b882cee5fa2e8423c2dff8edfe665c97d85ce48626c60931f27a9214c
Jul 30 17:45:00.302286 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:00.302269    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 17:45:00.787610 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:00.787575    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757225-7lrcp" event={"ID":"940576c4-8056-4abe-99d1-253cefffb3c3","Type":"ContainerStarted","Data":"72eaee0b882cee5fa2e8423c2dff8edfe665c97d85ce48626c60931f27a9214c"}
Jul 30 17:45:02.801890 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:02.801856    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757225-7lrcp" event={"ID":"940576c4-8056-4abe-99d1-253cefffb3c3","Type":"ContainerStarted","Data":"0cafbb5b549bc24f6dd31f5855cea601119fda35f15ba428737eb6e8e73a8a57"}
Jul 30 17:45:02.815610 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:02.815561    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757225-7lrcp" podStartSLOduration=0.68197226 podStartE2EDuration="2.815546369s" totalImagesPullingTime="2.133574109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:45:02.814415085 +0000 UTC m=+1077.267979076" watchObservedRunningTime="2026-07-30 17:45:02.815546369 +0000 UTC m=+1077.269110361"
Jul 30 17:45:03.807803 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:03.807764    2577 generic.go:356] "Generic (PLEG): container finished" podID="940576c4-8056-4abe-99d1-253cefffb3c3" containerID="0cafbb5b549bc24f6dd31f5855cea601119fda35f15ba428737eb6e8e73a8a57" exitCode=0
Jul 30 17:45:03.808218 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:03.807851    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757225-7lrcp" event={"ID":"940576c4-8056-4abe-99d1-253cefffb3c3","Type":"ContainerDied","Data":"0cafbb5b549bc24f6dd31f5855cea601119fda35f15ba428737eb6e8e73a8a57"}
Jul 30 17:45:04.950385 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:04.950362    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757225-7lrcp"
Jul 30 17:45:05.819483 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:05.819451    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757225-7lrcp" event={"ID":"940576c4-8056-4abe-99d1-253cefffb3c3","Type":"ContainerDied","Data":"72eaee0b882cee5fa2e8423c2dff8edfe665c97d85ce48626c60931f27a9214c"}
Jul 30 17:45:05.819483 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:05.819481    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757225-7lrcp"
Jul 30 17:45:05.819696 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:05.819497    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="72eaee0b882cee5fa2e8423c2dff8edfe665c97d85ce48626c60931f27a9214c"
Jul 30 17:45:13.039524 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:13.039487    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-59fc4686ff-d7sxw"]
Jul 30 17:45:13.039939 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:13.039911    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="940576c4-8056-4abe-99d1-253cefffb3c3" containerName="cleanup"
Jul 30 17:45:13.039939 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:13.039923    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="940576c4-8056-4abe-99d1-253cefffb3c3" containerName="cleanup"
Jul 30 17:45:13.040013 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:13.039998    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="940576c4-8056-4abe-99d1-253cefffb3c3" containerName="cleanup"
Jul 30 17:45:13.043158 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:13.043142    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59fc4686ff-d7sxw"
Jul 30 17:45:13.046235 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:13.046206    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 30 17:45:13.046396 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:13.046267    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 30 17:45:13.046396 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:13.046270    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-n5rvf\""
Jul 30 17:45:13.052218 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:13.052193    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59fc4686ff-d7sxw"]
Jul 30 17:45:13.104692 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:13.104655    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbhtg\" (UniqueName: \"kubernetes.io/projected/233aaa75-60f9-41c2-b701-466cf8d0f8b7-kube-api-access-qbhtg\") pod \"maas-controller-59fc4686ff-d7sxw\" (UID: \"233aaa75-60f9-41c2-b701-466cf8d0f8b7\") " pod="opendatahub/maas-controller-59fc4686ff-d7sxw"
Jul 30 17:45:13.104692 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:13.104697    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/233aaa75-60f9-41c2-b701-466cf8d0f8b7-webhook-cert\") pod \"maas-controller-59fc4686ff-d7sxw\" (UID: \"233aaa75-60f9-41c2-b701-466cf8d0f8b7\") " pod="opendatahub/maas-controller-59fc4686ff-d7sxw"
Jul 30 17:45:13.206056 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:13.206017    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbhtg\" (UniqueName: \"kubernetes.io/projected/233aaa75-60f9-41c2-b701-466cf8d0f8b7-kube-api-access-qbhtg\") pod \"maas-controller-59fc4686ff-d7sxw\" (UID: \"233aaa75-60f9-41c2-b701-466cf8d0f8b7\") " pod="opendatahub/maas-controller-59fc4686ff-d7sxw"
Jul 30 17:45:13.206269 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:13.206079    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/233aaa75-60f9-41c2-b701-466cf8d0f8b7-webhook-cert\") pod \"maas-controller-59fc4686ff-d7sxw\" (UID: \"233aaa75-60f9-41c2-b701-466cf8d0f8b7\") " pod="opendatahub/maas-controller-59fc4686ff-d7sxw"
Jul 30 17:45:13.208748 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:13.208725    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/233aaa75-60f9-41c2-b701-466cf8d0f8b7-webhook-cert\") pod \"maas-controller-59fc4686ff-d7sxw\" (UID: \"233aaa75-60f9-41c2-b701-466cf8d0f8b7\") " pod="opendatahub/maas-controller-59fc4686ff-d7sxw"
Jul 30 17:45:13.214313 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:13.214295    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbhtg\" (UniqueName: \"kubernetes.io/projected/233aaa75-60f9-41c2-b701-466cf8d0f8b7-kube-api-access-qbhtg\") pod \"maas-controller-59fc4686ff-d7sxw\" (UID: \"233aaa75-60f9-41c2-b701-466cf8d0f8b7\") " pod="opendatahub/maas-controller-59fc4686ff-d7sxw"
Jul 30 17:45:13.354874 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:13.354752    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59fc4686ff-d7sxw"
Jul 30 17:45:13.491366 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:13.491343    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59fc4686ff-d7sxw"]
Jul 30 17:45:13.493298 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:45:13.493271    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod233aaa75_60f9_41c2_b701_466cf8d0f8b7.slice/crio-f1cfc33930664a1e85a0ea8741fda5b880514a6353397f1f56bc8509da40146b WatchSource:0}: Error finding container f1cfc33930664a1e85a0ea8741fda5b880514a6353397f1f56bc8509da40146b: Status 404 returned error can't find the container with id f1cfc33930664a1e85a0ea8741fda5b880514a6353397f1f56bc8509da40146b
Jul 30 17:45:13.860297 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:13.860258    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59fc4686ff-d7sxw" event={"ID":"233aaa75-60f9-41c2-b701-466cf8d0f8b7","Type":"ContainerStarted","Data":"f1cfc33930664a1e85a0ea8741fda5b880514a6353397f1f56bc8509da40146b"}
Jul 30 17:45:14.867057 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:14.867008    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59fc4686ff-d7sxw" event={"ID":"233aaa75-60f9-41c2-b701-466cf8d0f8b7","Type":"ContainerStarted","Data":"9edeff253085c8d7acc14b79932ab2038003e60300afd570bd4eb83c426f6ac8"}
Jul 30 17:45:14.867588 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:14.867086    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-59fc4686ff-d7sxw"
Jul 30 17:45:14.888576 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:14.888532    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-59fc4686ff-d7sxw" podStartSLOduration=1.3424192050000001 podStartE2EDuration="1.888517566s" totalImagesPullingTime="546.098361ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:45:14.886504523 +0000 UTC m=+1089.340068513" watchObservedRunningTime="2026-07-30 17:45:14.888517566 +0000 UTC m=+1089.342081603"
Jul 30 17:45:25.874987 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:25.874958    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-59fc4686ff-d7sxw"
Jul 30 17:45:30.447950 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:30.447921    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:45:30.568251 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:30.568218    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-59fc4686ff-d7sxw"]
Jul 30 17:45:30.568445 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:30.568409    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-59fc4686ff-d7sxw" podUID="233aaa75-60f9-41c2-b701-466cf8d0f8b7" containerName="manager" containerID="cri-o://9edeff253085c8d7acc14b79932ab2038003e60300afd570bd4eb83c426f6ac8" gracePeriod=30
Jul 30 17:45:30.825318 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:30.825291    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59fc4686ff-d7sxw"
Jul 30 17:45:30.945635 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:30.945599    2577 generic.go:356] "Generic (PLEG): container finished" podID="233aaa75-60f9-41c2-b701-466cf8d0f8b7" containerID="9edeff253085c8d7acc14b79932ab2038003e60300afd570bd4eb83c426f6ac8" exitCode=0
Jul 30 17:45:30.945807 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:30.945645    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59fc4686ff-d7sxw"
Jul 30 17:45:30.945807 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:30.945656    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59fc4686ff-d7sxw" event={"ID":"233aaa75-60f9-41c2-b701-466cf8d0f8b7","Type":"ContainerDied","Data":"9edeff253085c8d7acc14b79932ab2038003e60300afd570bd4eb83c426f6ac8"}
Jul 30 17:45:30.945807 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:30.945712    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59fc4686ff-d7sxw" event={"ID":"233aaa75-60f9-41c2-b701-466cf8d0f8b7","Type":"ContainerDied","Data":"f1cfc33930664a1e85a0ea8741fda5b880514a6353397f1f56bc8509da40146b"}
Jul 30 17:45:30.945807 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:30.945742    2577 scope.go:117] "RemoveContainer" containerID="9edeff253085c8d7acc14b79932ab2038003e60300afd570bd4eb83c426f6ac8"
Jul 30 17:45:30.955043 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:30.955024    2577 scope.go:117] "RemoveContainer" containerID="9edeff253085c8d7acc14b79932ab2038003e60300afd570bd4eb83c426f6ac8"
Jul 30 17:45:30.955351 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:45:30.955333    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9edeff253085c8d7acc14b79932ab2038003e60300afd570bd4eb83c426f6ac8\": container with ID starting with 9edeff253085c8d7acc14b79932ab2038003e60300afd570bd4eb83c426f6ac8 not found: ID does not exist" containerID="9edeff253085c8d7acc14b79932ab2038003e60300afd570bd4eb83c426f6ac8"
Jul 30 17:45:30.955396 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:30.955361    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9edeff253085c8d7acc14b79932ab2038003e60300afd570bd4eb83c426f6ac8"} err="failed to get container status \"9edeff253085c8d7acc14b79932ab2038003e60300afd570bd4eb83c426f6ac8\": rpc error: code = NotFound desc = could not find container \"9edeff253085c8d7acc14b79932ab2038003e60300afd570bd4eb83c426f6ac8\": container with ID starting with 9edeff253085c8d7acc14b79932ab2038003e60300afd570bd4eb83c426f6ac8 not found: ID does not exist"
Jul 30 17:45:30.969249 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:30.969228    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbhtg\" (UniqueName: \"kubernetes.io/projected/233aaa75-60f9-41c2-b701-466cf8d0f8b7-kube-api-access-qbhtg\") pod \"233aaa75-60f9-41c2-b701-466cf8d0f8b7\" (UID: \"233aaa75-60f9-41c2-b701-466cf8d0f8b7\") "
Jul 30 17:45:30.969402 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:30.969388    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/233aaa75-60f9-41c2-b701-466cf8d0f8b7-webhook-cert\") pod \"233aaa75-60f9-41c2-b701-466cf8d0f8b7\" (UID: \"233aaa75-60f9-41c2-b701-466cf8d0f8b7\") "
Jul 30 17:45:30.971392 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:30.971367    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/233aaa75-60f9-41c2-b701-466cf8d0f8b7-kube-api-access-qbhtg" (OuterVolumeSpecName: "kube-api-access-qbhtg") pod "233aaa75-60f9-41c2-b701-466cf8d0f8b7" (UID: "233aaa75-60f9-41c2-b701-466cf8d0f8b7"). InnerVolumeSpecName "kube-api-access-qbhtg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:45:30.971506 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:30.971484    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/233aaa75-60f9-41c2-b701-466cf8d0f8b7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "233aaa75-60f9-41c2-b701-466cf8d0f8b7" (UID: "233aaa75-60f9-41c2-b701-466cf8d0f8b7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 17:45:31.070310 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:31.070267    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbhtg\" (UniqueName: \"kubernetes.io/projected/233aaa75-60f9-41c2-b701-466cf8d0f8b7-kube-api-access-qbhtg\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:45:31.070310 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:31.070308    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/233aaa75-60f9-41c2-b701-466cf8d0f8b7-webhook-cert\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:45:31.268511 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:31.268475    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-59fc4686ff-d7sxw"]
Jul 30 17:45:31.270325 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:31.270299    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-59fc4686ff-d7sxw"]
Jul 30 17:45:31.995391 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:31.995361    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:45:32.098850 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:32.098800    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-59fc4686ff-gfw5x"]
Jul 30 17:45:32.099303 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:32.099284    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="233aaa75-60f9-41c2-b701-466cf8d0f8b7" containerName="manager"
Jul 30 17:45:32.099395 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:32.099304    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="233aaa75-60f9-41c2-b701-466cf8d0f8b7" containerName="manager"
Jul 30 17:45:32.099449 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:32.099420    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="233aaa75-60f9-41c2-b701-466cf8d0f8b7" containerName="manager"
Jul 30 17:45:32.102603 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:32.102581    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59fc4686ff-gfw5x"
Jul 30 17:45:32.105553 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:32.105532    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-n5rvf\""
Jul 30 17:45:32.105690 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:32.105532    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 30 17:45:32.105690 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:32.105533    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 30 17:45:32.112944 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:32.112899    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="233aaa75-60f9-41c2-b701-466cf8d0f8b7" path="/var/lib/kubelet/pods/233aaa75-60f9-41c2-b701-466cf8d0f8b7/volumes"
Jul 30 17:45:32.113878 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:32.113858    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59fc4686ff-gfw5x"]
Jul 30 17:45:32.180575 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:32.180541    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px5ff\" (UniqueName: \"kubernetes.io/projected/170f7d7d-61e4-4269-8884-fae812916c3e-kube-api-access-px5ff\") pod \"maas-controller-59fc4686ff-gfw5x\" (UID: \"170f7d7d-61e4-4269-8884-fae812916c3e\") " pod="opendatahub/maas-controller-59fc4686ff-gfw5x"
Jul 30 17:45:32.180756 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:32.180665    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/170f7d7d-61e4-4269-8884-fae812916c3e-webhook-cert\") pod \"maas-controller-59fc4686ff-gfw5x\" (UID: \"170f7d7d-61e4-4269-8884-fae812916c3e\") " pod="opendatahub/maas-controller-59fc4686ff-gfw5x"
Jul 30 17:45:32.281385 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:32.281351    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/170f7d7d-61e4-4269-8884-fae812916c3e-webhook-cert\") pod \"maas-controller-59fc4686ff-gfw5x\" (UID: \"170f7d7d-61e4-4269-8884-fae812916c3e\") " pod="opendatahub/maas-controller-59fc4686ff-gfw5x"
Jul 30 17:45:32.281555 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:32.281392    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px5ff\" (UniqueName: \"kubernetes.io/projected/170f7d7d-61e4-4269-8884-fae812916c3e-kube-api-access-px5ff\") pod \"maas-controller-59fc4686ff-gfw5x\" (UID: \"170f7d7d-61e4-4269-8884-fae812916c3e\") " pod="opendatahub/maas-controller-59fc4686ff-gfw5x"
Jul 30 17:45:32.284005 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:32.283980    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/170f7d7d-61e4-4269-8884-fae812916c3e-webhook-cert\") pod \"maas-controller-59fc4686ff-gfw5x\" (UID: \"170f7d7d-61e4-4269-8884-fae812916c3e\") " pod="opendatahub/maas-controller-59fc4686ff-gfw5x"
Jul 30 17:45:32.289537 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:32.289512    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px5ff\" (UniqueName: \"kubernetes.io/projected/170f7d7d-61e4-4269-8884-fae812916c3e-kube-api-access-px5ff\") pod \"maas-controller-59fc4686ff-gfw5x\" (UID: \"170f7d7d-61e4-4269-8884-fae812916c3e\") " pod="opendatahub/maas-controller-59fc4686ff-gfw5x"
Jul 30 17:45:32.414111 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:32.414085    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59fc4686ff-gfw5x"
Jul 30 17:45:32.553050 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:32.553024    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59fc4686ff-gfw5x"]
Jul 30 17:45:32.554676 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:45:32.554641    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod170f7d7d_61e4_4269_8884_fae812916c3e.slice/crio-d9af3d6eb1628a848feb4c4b6df870ca09bc87fe3fd0bb49fedff512f7299970 WatchSource:0}: Error finding container d9af3d6eb1628a848feb4c4b6df870ca09bc87fe3fd0bb49fedff512f7299970: Status 404 returned error can't find the container with id d9af3d6eb1628a848feb4c4b6df870ca09bc87fe3fd0bb49fedff512f7299970
Jul 30 17:45:32.959923 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:32.959777    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59fc4686ff-gfw5x" event={"ID":"170f7d7d-61e4-4269-8884-fae812916c3e","Type":"ContainerStarted","Data":"79816ceca7256c9ee74d42ab8b880e2caad809d00283d844c60b0562d506b19e"}
Jul 30 17:45:32.959923 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:32.959866    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-59fc4686ff-gfw5x"
Jul 30 17:45:32.959923 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:32.959889    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59fc4686ff-gfw5x" event={"ID":"170f7d7d-61e4-4269-8884-fae812916c3e","Type":"ContainerStarted","Data":"d9af3d6eb1628a848feb4c4b6df870ca09bc87fe3fd0bb49fedff512f7299970"}
Jul 30 17:45:32.984279 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:32.984229    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-59fc4686ff-gfw5x" podStartSLOduration=0.657314652 podStartE2EDuration="984.214453ms" totalImagesPullingTime="326.899801ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:45:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:45:32.981706205 +0000 UTC m=+1107.435270195" watchObservedRunningTime="2026-07-30 17:45:32.984214453 +0000 UTC m=+1107.437778443"
Jul 30 17:45:43.968227 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:45:43.968198    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-59fc4686ff-gfw5x"
Jul 30 17:46:27.803497 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:46:27.803413    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:46:27.901771 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:46:27.901733    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:46:28.022110 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:46:28.022074    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:46:34.466389 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:46:34.466352    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:46:34.590311 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:46:34.590280    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:46:34.746279 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:46:34.746246    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:46:34.886211 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:46:34.886181    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:46:41.071441 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:46:41.071399    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:46:41.147368 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:46:41.147331    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:47:30.725850 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:47:30.725785    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:49:25.301145 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:49:25.301112    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:50:21.710842 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:50:21.710789    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:50:21.784078 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:50:21.784045    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:52:45.761235 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:52:45.761191    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t"]
Jul 30 17:52:45.761856 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:52:45.761614    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t" podUID="1d59e94d-29f3-48ba-9ae8-d8d0de63e124" containerName="manager" containerID="cri-o://f53ffa6dd3b6161a4a4a40ed4464f20909923b8de54ae3830cb4d735b291a657" gracePeriod=10
Jul 30 17:52:46.015067 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:52:46.014992    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t"
Jul 30 17:52:46.149631 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:52:46.149592    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bgpm\" (UniqueName: \"kubernetes.io/projected/1d59e94d-29f3-48ba-9ae8-d8d0de63e124-kube-api-access-4bgpm\") pod \"1d59e94d-29f3-48ba-9ae8-d8d0de63e124\" (UID: \"1d59e94d-29f3-48ba-9ae8-d8d0de63e124\") "
Jul 30 17:52:46.149631 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:52:46.149636    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d59e94d-29f3-48ba-9ae8-d8d0de63e124-extensions-socket-volume\") pod \"1d59e94d-29f3-48ba-9ae8-d8d0de63e124\" (UID: \"1d59e94d-29f3-48ba-9ae8-d8d0de63e124\") "
Jul 30 17:52:46.150082 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:52:46.150061    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d59e94d-29f3-48ba-9ae8-d8d0de63e124-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1d59e94d-29f3-48ba-9ae8-d8d0de63e124" (UID: "1d59e94d-29f3-48ba-9ae8-d8d0de63e124"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 17:52:46.151846 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:52:46.151811    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d59e94d-29f3-48ba-9ae8-d8d0de63e124-kube-api-access-4bgpm" (OuterVolumeSpecName: "kube-api-access-4bgpm") pod "1d59e94d-29f3-48ba-9ae8-d8d0de63e124" (UID: "1d59e94d-29f3-48ba-9ae8-d8d0de63e124"). InnerVolumeSpecName "kube-api-access-4bgpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 17:52:46.172259 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:52:46.172231    2577 generic.go:356] "Generic (PLEG): container finished" podID="1d59e94d-29f3-48ba-9ae8-d8d0de63e124" containerID="f53ffa6dd3b6161a4a4a40ed4464f20909923b8de54ae3830cb4d735b291a657" exitCode=0
Jul 30 17:52:46.172411 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:52:46.172276    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t" event={"ID":"1d59e94d-29f3-48ba-9ae8-d8d0de63e124","Type":"ContainerDied","Data":"f53ffa6dd3b6161a4a4a40ed4464f20909923b8de54ae3830cb4d735b291a657"}
Jul 30 17:52:46.172411 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:52:46.172284    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t"
Jul 30 17:52:46.172411 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:52:46.172318    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t" event={"ID":"1d59e94d-29f3-48ba-9ae8-d8d0de63e124","Type":"ContainerDied","Data":"5e59fe545b8fc22a05f23fd315d55737f537e7fb42a9a813960fb348de31c7bc"}
Jul 30 17:52:46.172411 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:52:46.172341    2577 scope.go:117] "RemoveContainer" containerID="f53ffa6dd3b6161a4a4a40ed4464f20909923b8de54ae3830cb4d735b291a657"
Jul 30 17:52:46.183616 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:52:46.183593    2577 scope.go:117] "RemoveContainer" containerID="f53ffa6dd3b6161a4a4a40ed4464f20909923b8de54ae3830cb4d735b291a657"
Jul 30 17:52:46.183902 ip-10-0-135-203 kubenswrapper[2577]: E0730 17:52:46.183882    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f53ffa6dd3b6161a4a4a40ed4464f20909923b8de54ae3830cb4d735b291a657\": container with ID starting with f53ffa6dd3b6161a4a4a40ed4464f20909923b8de54ae3830cb4d735b291a657 not found: ID does not exist" containerID="f53ffa6dd3b6161a4a4a40ed4464f20909923b8de54ae3830cb4d735b291a657"
Jul 30 17:52:46.183964 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:52:46.183912    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f53ffa6dd3b6161a4a4a40ed4464f20909923b8de54ae3830cb4d735b291a657"} err="failed to get container status \"f53ffa6dd3b6161a4a4a40ed4464f20909923b8de54ae3830cb4d735b291a657\": rpc error: code = NotFound desc = could not find container \"f53ffa6dd3b6161a4a4a40ed4464f20909923b8de54ae3830cb4d735b291a657\": container with ID starting with f53ffa6dd3b6161a4a4a40ed4464f20909923b8de54ae3830cb4d735b291a657 not found: ID does not exist"
Jul 30 17:52:46.195128 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:52:46.195100    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t"]
Jul 30 17:52:46.198693 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:52:46.198669    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhr6t"]
Jul 30 17:52:46.251320 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:52:46.251277    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bgpm\" (UniqueName: \"kubernetes.io/projected/1d59e94d-29f3-48ba-9ae8-d8d0de63e124-kube-api-access-4bgpm\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:52:46.251320 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:52:46.251318    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d59e94d-29f3-48ba-9ae8-d8d0de63e124-extensions-socket-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 17:52:48.110548 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:52:48.110515    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d59e94d-29f3-48ba-9ae8-d8d0de63e124" path="/var/lib/kubelet/pods/1d59e94d-29f3-48ba-9ae8-d8d0de63e124/volumes"
Jul 30 17:53:20.262649 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:53:20.262617    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:53:46.485834 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:53:46.485793    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l"]
Jul 30 17:53:46.488316 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:53:46.486209    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d59e94d-29f3-48ba-9ae8-d8d0de63e124" containerName="manager"
Jul 30 17:53:46.488316 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:53:46.486222    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d59e94d-29f3-48ba-9ae8-d8d0de63e124" containerName="manager"
Jul 30 17:53:46.488316 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:53:46.486285    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d59e94d-29f3-48ba-9ae8-d8d0de63e124" containerName="manager"
Jul 30 17:53:46.489165 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:53:46.489150    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l"
Jul 30 17:53:46.491563 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:53:46.491540    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-lw975\""
Jul 30 17:53:46.504207 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:53:46.504186    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l"]
Jul 30 17:53:46.512436 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:53:46.512405    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/60d5eb12-9d1d-4233-a4d9-f2cc305882c0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5rp4l\" (UID: \"60d5eb12-9d1d-4233-a4d9-f2cc305882c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l"
Jul 30 17:53:46.512565 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:53:46.512550    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcs75\" (UniqueName: \"kubernetes.io/projected/60d5eb12-9d1d-4233-a4d9-f2cc305882c0-kube-api-access-fcs75\") pod \"kuadrant-operator-controller-manager-86878c769f-5rp4l\" (UID: \"60d5eb12-9d1d-4233-a4d9-f2cc305882c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l"
Jul 30 17:53:46.613378 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:53:46.613328    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcs75\" (UniqueName: \"kubernetes.io/projected/60d5eb12-9d1d-4233-a4d9-f2cc305882c0-kube-api-access-fcs75\") pod \"kuadrant-operator-controller-manager-86878c769f-5rp4l\" (UID: \"60d5eb12-9d1d-4233-a4d9-f2cc305882c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l"
Jul 30 17:53:46.613613 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:53:46.613421    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/60d5eb12-9d1d-4233-a4d9-f2cc305882c0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5rp4l\" (UID: \"60d5eb12-9d1d-4233-a4d9-f2cc305882c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l"
Jul 30 17:53:46.613760 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:53:46.613744    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/60d5eb12-9d1d-4233-a4d9-f2cc305882c0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5rp4l\" (UID: \"60d5eb12-9d1d-4233-a4d9-f2cc305882c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l"
Jul 30 17:53:46.622278 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:53:46.622253    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcs75\" (UniqueName: \"kubernetes.io/projected/60d5eb12-9d1d-4233-a4d9-f2cc305882c0-kube-api-access-fcs75\") pod \"kuadrant-operator-controller-manager-86878c769f-5rp4l\" (UID: \"60d5eb12-9d1d-4233-a4d9-f2cc305882c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l"
Jul 30 17:53:46.798962 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:53:46.798934    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l"
Jul 30 17:53:46.946513 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:53:46.946451    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l"]
Jul 30 17:53:46.950030 ip-10-0-135-203 kubenswrapper[2577]: W0730 17:53:46.950000    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60d5eb12_9d1d_4233_a4d9_f2cc305882c0.slice/crio-e069563ea00856ac5393390382c45e3860b332e47aa10e70a6592823b371ab97 WatchSource:0}: Error finding container e069563ea00856ac5393390382c45e3860b332e47aa10e70a6592823b371ab97: Status 404 returned error can't find the container with id e069563ea00856ac5393390382c45e3860b332e47aa10e70a6592823b371ab97
Jul 30 17:53:47.487022 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:53:47.486980    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l" event={"ID":"60d5eb12-9d1d-4233-a4d9-f2cc305882c0","Type":"ContainerStarted","Data":"7c846d4de370f316ccc14b7607f8ab00b577a0bb4f3b43216d7e834a55d6667f"}
Jul 30 17:53:47.487423 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:53:47.487042    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l" event={"ID":"60d5eb12-9d1d-4233-a4d9-f2cc305882c0","Type":"ContainerStarted","Data":"e069563ea00856ac5393390382c45e3860b332e47aa10e70a6592823b371ab97"}
Jul 30 17:53:47.487423 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:53:47.487089    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l"
Jul 30 17:53:47.512504 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:53:47.512459    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l" podStartSLOduration=1.5124450710000001 podStartE2EDuration="1.512445071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 17:53:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 17:53:47.51011788 +0000 UTC m=+1601.963681870" watchObservedRunningTime="2026-07-30 17:53:47.512445071 +0000 UTC m=+1601.966009112"
Jul 30 17:53:58.494910 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:53:58.494832    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l"
Jul 30 17:55:31.584360 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:55:31.584322    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:55:31.667860 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:55:31.667810    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:57:01.749534 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:57:01.749440    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:57:10.553897 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:57:10.553848    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:57:25.906744 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:57:25.906712    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:57:42.267750 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:57:42.267718    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 17:57:43.235921 ip-10-0-135-203 kubenswrapper[2577]: I0730 17:57:43.235893    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:00:00.136499 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:00:00.136424    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757240-8fwlc"]
Jul 30 18:00:00.139745 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:00:00.139729    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757240-8fwlc"
Jul 30 18:00:00.142270 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:00:00.142254    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qw74k\""
Jul 30 18:00:00.149138 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:00:00.149123    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757240-8fwlc"
Jul 30 18:00:00.158160 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:00:00.158130    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757240-8fwlc"]
Jul 30 18:00:00.293581 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:00:00.293557    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757240-8fwlc"]
Jul 30 18:00:00.295630 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:00:00.295604    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod384eb3e0_9877_4a95_975a_49016d7a7fe0.slice/crio-ae1d94776d05dc8b227d5dffdf0205fa84dce4777c18d2eccd9199968df5f448 WatchSource:0}: Error finding container ae1d94776d05dc8b227d5dffdf0205fa84dce4777c18d2eccd9199968df5f448: Status 404 returned error can't find the container with id ae1d94776d05dc8b227d5dffdf0205fa84dce4777c18d2eccd9199968df5f448
Jul 30 18:00:00.387959 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:00:00.387872    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757240-8fwlc" event={"ID":"384eb3e0-9877-4a95-975a-49016d7a7fe0","Type":"ContainerStarted","Data":"ae1d94776d05dc8b227d5dffdf0205fa84dce4777c18d2eccd9199968df5f448"}
Jul 30 18:00:01.394615 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:00:01.394573    2577 generic.go:356] "Generic (PLEG): container finished" podID="384eb3e0-9877-4a95-975a-49016d7a7fe0" containerID="f375bb2e0cfd15b4ca2da77fbb197e91ea8a59b1008c436fdbfaeeec03ef278f" exitCode=0
Jul 30 18:00:01.395046 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:00:01.394662    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757240-8fwlc" event={"ID":"384eb3e0-9877-4a95-975a-49016d7a7fe0","Type":"ContainerDied","Data":"f375bb2e0cfd15b4ca2da77fbb197e91ea8a59b1008c436fdbfaeeec03ef278f"}
Jul 30 18:00:02.537329 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:00:02.537305    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757240-8fwlc"
Jul 30 18:00:03.406462 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:00:03.406431    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757240-8fwlc"
Jul 30 18:00:03.406643 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:00:03.406465    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757240-8fwlc" event={"ID":"384eb3e0-9877-4a95-975a-49016d7a7fe0","Type":"ContainerDied","Data":"ae1d94776d05dc8b227d5dffdf0205fa84dce4777c18d2eccd9199968df5f448"}
Jul 30 18:00:03.406643 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:00:03.406514    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae1d94776d05dc8b227d5dffdf0205fa84dce4777c18d2eccd9199968df5f448"
Jul 30 18:01:39.393626 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:01:39.393590    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:02:16.325961 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:16.325927    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:02:19.055989 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:19.055940    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:02:19.154170 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:19.154137    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:02:19.514928 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:19.514896    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:02:23.467097 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:23.467065    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:02:25.688950 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:25.688889    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:02:25.792191 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:25.792161    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:02:25.922219 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:25.922190    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:02:26.247836 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:26.247784    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:02:26.751927 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:26.751898    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:02:29.040282 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:29.040253    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:02:29.133677 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:29.133643    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:02:30.167090 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.167051    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57"]
Jul 30 18:02:30.167684 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.167667    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="384eb3e0-9877-4a95-975a-49016d7a7fe0" containerName="cleanup"
Jul 30 18:02:30.167755 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.167687    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="384eb3e0-9877-4a95-975a-49016d7a7fe0" containerName="cleanup"
Jul 30 18:02:30.167840 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.167806    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="384eb3e0-9877-4a95-975a-49016d7a7fe0" containerName="cleanup"
Jul 30 18:02:30.171184 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.171165    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57"
Jul 30 18:02:30.173377 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.173358    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-8a48bf8b-serving-cert\""
Jul 30 18:02:30.181754 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.181720    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57"]
Jul 30 18:02:30.229120 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.229091    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:02:30.289182 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.289141    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6-maas-api-tls\") pod \"maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57\" (UID: \"18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57"
Jul 30 18:02:30.289357 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.289190    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ztgw\" (UniqueName: \"kubernetes.io/projected/18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6-kube-api-access-9ztgw\") pod \"maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57\" (UID: \"18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57"
Jul 30 18:02:30.289357 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.289272    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6-openshift-service-ca\") pod \"maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57\" (UID: \"18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57"
Jul 30 18:02:30.390363 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.390323    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6-maas-api-tls\") pod \"maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57\" (UID: \"18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57"
Jul 30 18:02:30.390563 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.390395    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ztgw\" (UniqueName: \"kubernetes.io/projected/18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6-kube-api-access-9ztgw\") pod \"maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57\" (UID: \"18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57"
Jul 30 18:02:30.390563 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.390445    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6-openshift-service-ca\") pod \"maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57\" (UID: \"18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57"
Jul 30 18:02:30.390563 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:02:30.390491    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-serving-cert: secret "maas-api-e2e-ait-8a48bf8b-serving-cert" not found
Jul 30 18:02:30.390732 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:02:30.390569    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6-maas-api-tls podName:18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6 nodeName:}" failed. No retries permitted until 2026-07-30 18:02:30.890549998 +0000 UTC m=+2125.344113971 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6-maas-api-tls") pod "maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57" (UID: "18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6") : secret "maas-api-e2e-ait-8a48bf8b-serving-cert" not found
Jul 30 18:02:30.391202 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.391178    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6-openshift-service-ca\") pod \"maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57\" (UID: \"18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57"
Jul 30 18:02:30.404557 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.404529    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ztgw\" (UniqueName: \"kubernetes.io/projected/18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6-kube-api-access-9ztgw\") pod \"maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57\" (UID: \"18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57"
Jul 30 18:02:30.718004 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.717968    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:02:30.844181 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.844149    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:02:30.896217 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.896185    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6-maas-api-tls\") pod \"maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57\" (UID: \"18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57"
Jul 30 18:02:30.898682 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.898653    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6-maas-api-tls\") pod \"maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57\" (UID: \"18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57"
Jul 30 18:02:30.947031 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.946994    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:02:30.963453 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.963424    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc"]
Jul 30 18:02:30.967308 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.967291    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc"
Jul 30 18:02:30.973464 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.973401    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-8a48bf8b\""
Jul 30 18:02:30.973464 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.973412    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-8a48bf8b-dockercfg-zrrwq\""
Jul 30 18:02:30.980638 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:30.980608    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc"]
Jul 30 18:02:31.026465 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:31.026430    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6"]
Jul 30 18:02:31.029928 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:31.029911    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6"
Jul 30 18:02:31.038927 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:31.038896    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6"]
Jul 30 18:02:31.083871 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:31.083842    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57"
Jul 30 18:02:31.098290 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:31.098257    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a452f4e2-2682-49fe-b96a-2a0d60463409-plugins-config-volume\") pod \"payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc\" (UID: \"a452f4e2-2682-49fe-b96a-2a0d60463409\") " pod="openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc"
Jul 30 18:02:31.098501 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:31.098471    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zns9\" (UniqueName: \"kubernetes.io/projected/0b32af3a-36a6-4364-89b3-d206a847cf92-kube-api-access-6zns9\") pod \"payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6\" (UID: \"0b32af3a-36a6-4364-89b3-d206a847cf92\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6"
Jul 30 18:02:31.098643 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:31.098555    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qbzf\" (UniqueName: \"kubernetes.io/projected/a452f4e2-2682-49fe-b96a-2a0d60463409-kube-api-access-8qbzf\") pod \"payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc\" (UID: \"a452f4e2-2682-49fe-b96a-2a0d60463409\") " pod="openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc"
Jul 30 18:02:31.098643 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:31.098626    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b32af3a-36a6-4364-89b3-d206a847cf92-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6\" (UID: \"0b32af3a-36a6-4364-89b3-d206a847cf92\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6"
Jul 30 18:02:31.199941 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:31.199879    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zns9\" (UniqueName: \"kubernetes.io/projected/0b32af3a-36a6-4364-89b3-d206a847cf92-kube-api-access-6zns9\") pod \"payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6\" (UID: \"0b32af3a-36a6-4364-89b3-d206a847cf92\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6"
Jul 30 18:02:31.200397 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:31.199963    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qbzf\" (UniqueName: \"kubernetes.io/projected/a452f4e2-2682-49fe-b96a-2a0d60463409-kube-api-access-8qbzf\") pod \"payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc\" (UID: \"a452f4e2-2682-49fe-b96a-2a0d60463409\") " pod="openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc"
Jul 30 18:02:31.200397 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:31.200045    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b32af3a-36a6-4364-89b3-d206a847cf92-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6\" (UID: \"0b32af3a-36a6-4364-89b3-d206a847cf92\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6"
Jul 30 18:02:31.200397 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:31.200133    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a452f4e2-2682-49fe-b96a-2a0d60463409-plugins-config-volume\") pod \"payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc\" (UID: \"a452f4e2-2682-49fe-b96a-2a0d60463409\") " pod="openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc"
Jul 30 18:02:31.200765 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:31.200740    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b32af3a-36a6-4364-89b3-d206a847cf92-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6\" (UID: \"0b32af3a-36a6-4364-89b3-d206a847cf92\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6"
Jul 30 18:02:31.201308 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:31.201265    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a452f4e2-2682-49fe-b96a-2a0d60463409-plugins-config-volume\") pod \"payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc\" (UID: \"a452f4e2-2682-49fe-b96a-2a0d60463409\") " pod="openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc"
Jul 30 18:02:31.208644 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:31.208618    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qbzf\" (UniqueName: \"kubernetes.io/projected/a452f4e2-2682-49fe-b96a-2a0d60463409-kube-api-access-8qbzf\") pod \"payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc\" (UID: \"a452f4e2-2682-49fe-b96a-2a0d60463409\") " pod="openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc"
Jul 30 18:02:31.208920 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:31.208897    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zns9\" (UniqueName: \"kubernetes.io/projected/0b32af3a-36a6-4364-89b3-d206a847cf92-kube-api-access-6zns9\") pod \"payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6\" (UID: \"0b32af3a-36a6-4364-89b3-d206a847cf92\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6"
Jul 30 18:02:31.233266 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:31.233100    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57"]
Jul 30 18:02:31.235390 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:02:31.235366    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18aed62e_ba22_4ab2_99ae_5cf0cb4baaa6.slice/crio-c124abca96a94fa3e4672ea089f9f178eaa8210219abd9237de4b42f2bac6664 WatchSource:0}: Error finding container c124abca96a94fa3e4672ea089f9f178eaa8210219abd9237de4b42f2bac6664: Status 404 returned error can't find the container with id c124abca96a94fa3e4672ea089f9f178eaa8210219abd9237de4b42f2bac6664
Jul 30 18:02:31.236690 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:31.236671    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 18:02:31.279056 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:31.279021    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc"
Jul 30 18:02:31.342688 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:31.342276    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6"
Jul 30 18:02:31.434405 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:31.434371    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc"]
Jul 30 18:02:31.435878 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:02:31.435847    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda452f4e2_2682_49fe_b96a_2a0d60463409.slice/crio-9d4994c5c61785973f7e939eccdb8c0956aabdd4a528cdb1bec5ca88df2af177 WatchSource:0}: Error finding container 9d4994c5c61785973f7e939eccdb8c0956aabdd4a528cdb1bec5ca88df2af177: Status 404 returned error can't find the container with id 9d4994c5c61785973f7e939eccdb8c0956aabdd4a528cdb1bec5ca88df2af177
Jul 30 18:02:31.503897 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:31.503869    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6"]
Jul 30 18:02:31.507146 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:02:31.507119    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b32af3a_36a6_4364_89b3_d206a847cf92.slice/crio-36f7824bf562ec1d03455cd24560dc49663327c3f69af5622ed6840959b709da WatchSource:0}: Error finding container 36f7824bf562ec1d03455cd24560dc49663327c3f69af5622ed6840959b709da: Status 404 returned error can't find the container with id 36f7824bf562ec1d03455cd24560dc49663327c3f69af5622ed6840959b709da
Jul 30 18:02:32.163859 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:32.163806    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc" event={"ID":"a452f4e2-2682-49fe-b96a-2a0d60463409","Type":"ContainerStarted","Data":"62d75b2803966f4107b15f8dddcdf3685277d1cb343d51608be998b780005bc5"}
Jul 30 18:02:32.164156 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:32.164130    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc" event={"ID":"a452f4e2-2682-49fe-b96a-2a0d60463409","Type":"ContainerStarted","Data":"9d4994c5c61785973f7e939eccdb8c0956aabdd4a528cdb1bec5ca88df2af177"}
Jul 30 18:02:32.164515 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:32.164489    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc"
Jul 30 18:02:32.166620 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:32.166596    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6" event={"ID":"0b32af3a-36a6-4364-89b3-d206a847cf92","Type":"ContainerStarted","Data":"5ad59d1e33c864678d781ede7a42d2b0742f6400645b170f241376f1d98db2b6"}
Jul 30 18:02:32.166804 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:32.166789    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6" event={"ID":"0b32af3a-36a6-4364-89b3-d206a847cf92","Type":"ContainerStarted","Data":"36f7824bf562ec1d03455cd24560dc49663327c3f69af5622ed6840959b709da"}
Jul 30 18:02:32.166977 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:32.166955    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6"
Jul 30 18:02:32.168649 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:32.168624    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57" event={"ID":"18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6","Type":"ContainerStarted","Data":"eb2d3e4710a1cb8b5ef6f28cc73fe39aa68ac2f47714071334d93a3610034ce6"}
Jul 30 18:02:32.168750 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:32.168666    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57" event={"ID":"18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6","Type":"ContainerStarted","Data":"c124abca96a94fa3e4672ea089f9f178eaa8210219abd9237de4b42f2bac6664"}
Jul 30 18:02:32.168917 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:32.168888    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57"
Jul 30 18:02:32.181504 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:32.181405    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc" podStartSLOduration=2.181390608 podStartE2EDuration="2.181390608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:02:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:02:32.179632564 +0000 UTC m=+2126.633196555" watchObservedRunningTime="2026-07-30 18:02:32.181390608 +0000 UTC m=+2126.634954599"
Jul 30 18:02:32.196671 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:32.196156    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57" podStartSLOduration=1.512884169 podStartE2EDuration="2.196135315s" totalImagesPullingTime="683.251146ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:02:32.195037228 +0000 UTC m=+2126.648601218" watchObservedRunningTime="2026-07-30 18:02:32.196135315 +0000 UTC m=+2126.649699307"
Jul 30 18:02:32.210911 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:32.210857    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6" podStartSLOduration=1.210837788 podStartE2EDuration="1.210837788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:02:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:02:32.209225996 +0000 UTC m=+2126.662789988" watchObservedRunningTime="2026-07-30 18:02:32.210837788 +0000 UTC m=+2126.664401780"
Jul 30 18:02:32.500250 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:32.500200    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:02:32.655350 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:32.655320    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:02:38.180567 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:38.180538    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57"
Jul 30 18:02:39.282296 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:39.282264    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:02:39.383551 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:39.383515    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:02:43.177240 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:43.177202    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6" podUID="0b32af3a-36a6-4364-89b3-d206a847cf92" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.53:9004: connect: connection refused"
Jul 30 18:02:43.177803 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:43.177784    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc"
Jul 30 18:02:53.176179 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:02:53.176146    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6"
Jul 30 18:03:10.720619 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:10.720538    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57"]
Jul 30 18:03:10.721080 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:10.720761    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57" podUID="18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6" containerName="maas-api" containerID="cri-o://eb2d3e4710a1cb8b5ef6f28cc73fe39aa68ac2f47714071334d93a3610034ce6" gracePeriod=30
Jul 30 18:03:10.728460 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:10.728432    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:10.765248 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:10.765209    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc"]
Jul 30 18:03:10.765568 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:10.765513    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc" podUID="a452f4e2-2682-49fe-b96a-2a0d60463409" containerName="payload-processing" containerID="cri-o://62d75b2803966f4107b15f8dddcdf3685277d1cb343d51608be998b780005bc5" gracePeriod=30
Jul 30 18:03:10.773059 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:10.773027    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6"]
Jul 30 18:03:10.773259 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:10.773235    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6" podUID="0b32af3a-36a6-4364-89b3-d206a847cf92" containerName="payload-pre-processing" containerID="cri-o://5ad59d1e33c864678d781ede7a42d2b0742f6400645b170f241376f1d98db2b6" gracePeriod=30
Jul 30 18:03:10.814542 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:10.814505    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:10.876739 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:10.876703    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:10.941293 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:10.940855    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:11.039195 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.039037    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57"
Jul 30 18:03:11.084841 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.084793    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc"
Jul 30 18:03:11.089808 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.089783    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:11.096790 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.096769    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6"
Jul 30 18:03:11.184650 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.184618    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6-openshift-service-ca\") pod \"18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6\" (UID: \"18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6\") "
Jul 30 18:03:11.184872 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.184675    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qbzf\" (UniqueName: \"kubernetes.io/projected/a452f4e2-2682-49fe-b96a-2a0d60463409-kube-api-access-8qbzf\") pod \"a452f4e2-2682-49fe-b96a-2a0d60463409\" (UID: \"a452f4e2-2682-49fe-b96a-2a0d60463409\") "
Jul 30 18:03:11.184872 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.184732    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a452f4e2-2682-49fe-b96a-2a0d60463409-plugins-config-volume\") pod \"a452f4e2-2682-49fe-b96a-2a0d60463409\" (UID: \"a452f4e2-2682-49fe-b96a-2a0d60463409\") "
Jul 30 18:03:11.184872 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.184759    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b32af3a-36a6-4364-89b3-d206a847cf92-plugins-config-volume\") pod \"0b32af3a-36a6-4364-89b3-d206a847cf92\" (UID: \"0b32af3a-36a6-4364-89b3-d206a847cf92\") "
Jul 30 18:03:11.184872 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.184787    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6-maas-api-tls\") pod \"18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6\" (UID: \"18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6\") "
Jul 30 18:03:11.185114 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.185092    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a452f4e2-2682-49fe-b96a-2a0d60463409-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a452f4e2-2682-49fe-b96a-2a0d60463409" (UID: "a452f4e2-2682-49fe-b96a-2a0d60463409"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:03:11.185215 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.185116    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b32af3a-36a6-4364-89b3-d206a847cf92-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0b32af3a-36a6-4364-89b3-d206a847cf92" (UID: "0b32af3a-36a6-4364-89b3-d206a847cf92"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:03:11.185215 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.185133    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6" (UID: "18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:03:11.185295 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.185250    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ztgw\" (UniqueName: \"kubernetes.io/projected/18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6-kube-api-access-9ztgw\") pod \"18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6\" (UID: \"18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6\") "
Jul 30 18:03:11.185364 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.185348    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zns9\" (UniqueName: \"kubernetes.io/projected/0b32af3a-36a6-4364-89b3-d206a847cf92-kube-api-access-6zns9\") pod \"0b32af3a-36a6-4364-89b3-d206a847cf92\" (UID: \"0b32af3a-36a6-4364-89b3-d206a847cf92\") "
Jul 30 18:03:11.185875 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.185856    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a452f4e2-2682-49fe-b96a-2a0d60463409-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:03:11.185962 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.185880    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b32af3a-36a6-4364-89b3-d206a847cf92-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:03:11.185962 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.185895    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6-openshift-service-ca\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:03:11.187328 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.187297    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a452f4e2-2682-49fe-b96a-2a0d60463409-kube-api-access-8qbzf" (OuterVolumeSpecName: "kube-api-access-8qbzf") pod "a452f4e2-2682-49fe-b96a-2a0d60463409" (UID: "a452f4e2-2682-49fe-b96a-2a0d60463409"). InnerVolumeSpecName "kube-api-access-8qbzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:03:11.187328 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.187309    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6" (UID: "18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 18:03:11.187328 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.187319    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6-kube-api-access-9ztgw" (OuterVolumeSpecName: "kube-api-access-9ztgw") pod "18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6" (UID: "18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6"). InnerVolumeSpecName "kube-api-access-9ztgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:03:11.187519 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.187503    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b32af3a-36a6-4364-89b3-d206a847cf92-kube-api-access-6zns9" (OuterVolumeSpecName: "kube-api-access-6zns9") pod "0b32af3a-36a6-4364-89b3-d206a847cf92" (UID: "0b32af3a-36a6-4364-89b3-d206a847cf92"). InnerVolumeSpecName "kube-api-access-6zns9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:03:11.286658 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.286626    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zns9\" (UniqueName: \"kubernetes.io/projected/0b32af3a-36a6-4364-89b3-d206a847cf92-kube-api-access-6zns9\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:03:11.286658 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.286656    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qbzf\" (UniqueName: \"kubernetes.io/projected/a452f4e2-2682-49fe-b96a-2a0d60463409-kube-api-access-8qbzf\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:03:11.286885 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.286668    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6-maas-api-tls\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:03:11.286885 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.286677    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ztgw\" (UniqueName: \"kubernetes.io/projected/18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6-kube-api-access-9ztgw\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:03:11.320141 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.320105    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:11.374073 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.374034    2577 generic.go:356] "Generic (PLEG): container finished" podID="a452f4e2-2682-49fe-b96a-2a0d60463409" containerID="62d75b2803966f4107b15f8dddcdf3685277d1cb343d51608be998b780005bc5" exitCode=0
Jul 30 18:03:11.374261 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.374087    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc"
Jul 30 18:03:11.374261 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.374113    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc" event={"ID":"a452f4e2-2682-49fe-b96a-2a0d60463409","Type":"ContainerDied","Data":"62d75b2803966f4107b15f8dddcdf3685277d1cb343d51608be998b780005bc5"}
Jul 30 18:03:11.374261 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.374185    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc" event={"ID":"a452f4e2-2682-49fe-b96a-2a0d60463409","Type":"ContainerDied","Data":"9d4994c5c61785973f7e939eccdb8c0956aabdd4a528cdb1bec5ca88df2af177"}
Jul 30 18:03:11.374261 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.374222    2577 scope.go:117] "RemoveContainer" containerID="62d75b2803966f4107b15f8dddcdf3685277d1cb343d51608be998b780005bc5"
Jul 30 18:03:11.375671 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.375648    2577 generic.go:356] "Generic (PLEG): container finished" podID="0b32af3a-36a6-4364-89b3-d206a847cf92" containerID="5ad59d1e33c864678d781ede7a42d2b0742f6400645b170f241376f1d98db2b6" exitCode=0
Jul 30 18:03:11.375794 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.375694    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6"
Jul 30 18:03:11.375877 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.375795    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6" event={"ID":"0b32af3a-36a6-4364-89b3-d206a847cf92","Type":"ContainerDied","Data":"5ad59d1e33c864678d781ede7a42d2b0742f6400645b170f241376f1d98db2b6"}
Jul 30 18:03:11.375877 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.375858    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6" event={"ID":"0b32af3a-36a6-4364-89b3-d206a847cf92","Type":"ContainerDied","Data":"36f7824bf562ec1d03455cd24560dc49663327c3f69af5622ed6840959b709da"}
Jul 30 18:03:11.377358 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.377336    2577 generic.go:356] "Generic (PLEG): container finished" podID="18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6" containerID="eb2d3e4710a1cb8b5ef6f28cc73fe39aa68ac2f47714071334d93a3610034ce6" exitCode=0
Jul 30 18:03:11.377477 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.377380    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57"
Jul 30 18:03:11.377477 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.377401    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57" event={"ID":"18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6","Type":"ContainerDied","Data":"eb2d3e4710a1cb8b5ef6f28cc73fe39aa68ac2f47714071334d93a3610034ce6"}
Jul 30 18:03:11.377477 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.377444    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57" event={"ID":"18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6","Type":"ContainerDied","Data":"c124abca96a94fa3e4672ea089f9f178eaa8210219abd9237de4b42f2bac6664"}
Jul 30 18:03:11.384239 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.384219    2577 scope.go:117] "RemoveContainer" containerID="62d75b2803966f4107b15f8dddcdf3685277d1cb343d51608be998b780005bc5"
Jul 30 18:03:11.384533 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:03:11.384507    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62d75b2803966f4107b15f8dddcdf3685277d1cb343d51608be998b780005bc5\": container with ID starting with 62d75b2803966f4107b15f8dddcdf3685277d1cb343d51608be998b780005bc5 not found: ID does not exist" containerID="62d75b2803966f4107b15f8dddcdf3685277d1cb343d51608be998b780005bc5"
Jul 30 18:03:11.384603 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.384544    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62d75b2803966f4107b15f8dddcdf3685277d1cb343d51608be998b780005bc5"} err="failed to get container status \"62d75b2803966f4107b15f8dddcdf3685277d1cb343d51608be998b780005bc5\": rpc error: code = NotFound desc = could not find container \"62d75b2803966f4107b15f8dddcdf3685277d1cb343d51608be998b780005bc5\": container with ID starting with 62d75b2803966f4107b15f8dddcdf3685277d1cb343d51608be998b780005bc5 not found: ID does not exist"
Jul 30 18:03:11.384603 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.384564    2577 scope.go:117] "RemoveContainer" containerID="5ad59d1e33c864678d781ede7a42d2b0742f6400645b170f241376f1d98db2b6"
Jul 30 18:03:11.394490 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.394473    2577 scope.go:117] "RemoveContainer" containerID="5ad59d1e33c864678d781ede7a42d2b0742f6400645b170f241376f1d98db2b6"
Jul 30 18:03:11.394755 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:03:11.394736    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ad59d1e33c864678d781ede7a42d2b0742f6400645b170f241376f1d98db2b6\": container with ID starting with 5ad59d1e33c864678d781ede7a42d2b0742f6400645b170f241376f1d98db2b6 not found: ID does not exist" containerID="5ad59d1e33c864678d781ede7a42d2b0742f6400645b170f241376f1d98db2b6"
Jul 30 18:03:11.394852 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.394767    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ad59d1e33c864678d781ede7a42d2b0742f6400645b170f241376f1d98db2b6"} err="failed to get container status \"5ad59d1e33c864678d781ede7a42d2b0742f6400645b170f241376f1d98db2b6\": rpc error: code = NotFound desc = could not find container \"5ad59d1e33c864678d781ede7a42d2b0742f6400645b170f241376f1d98db2b6\": container with ID starting with 5ad59d1e33c864678d781ede7a42d2b0742f6400645b170f241376f1d98db2b6 not found: ID does not exist"
Jul 30 18:03:11.394852 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.394791    2577 scope.go:117] "RemoveContainer" containerID="eb2d3e4710a1cb8b5ef6f28cc73fe39aa68ac2f47714071334d93a3610034ce6"
Jul 30 18:03:11.402026 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.402004    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6"]
Jul 30 18:03:11.402927 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.402915    2577 scope.go:117] "RemoveContainer" containerID="eb2d3e4710a1cb8b5ef6f28cc73fe39aa68ac2f47714071334d93a3610034ce6"
Jul 30 18:03:11.403226 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:03:11.403197    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb2d3e4710a1cb8b5ef6f28cc73fe39aa68ac2f47714071334d93a3610034ce6\": container with ID starting with eb2d3e4710a1cb8b5ef6f28cc73fe39aa68ac2f47714071334d93a3610034ce6 not found: ID does not exist" containerID="eb2d3e4710a1cb8b5ef6f28cc73fe39aa68ac2f47714071334d93a3610034ce6"
Jul 30 18:03:11.403320 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.403229    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb2d3e4710a1cb8b5ef6f28cc73fe39aa68ac2f47714071334d93a3610034ce6"} err="failed to get container status \"eb2d3e4710a1cb8b5ef6f28cc73fe39aa68ac2f47714071334d93a3610034ce6\": rpc error: code = NotFound desc = could not find container \"eb2d3e4710a1cb8b5ef6f28cc73fe39aa68ac2f47714071334d93a3610034ce6\": container with ID starting with eb2d3e4710a1cb8b5ef6f28cc73fe39aa68ac2f47714071334d93a3610034ce6 not found: ID does not exist"
Jul 30 18:03:11.407390 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.407368    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8a48bf8b-f47bfcb6d-64gb6"]
Jul 30 18:03:11.417616 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.417595    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc"]
Jul 30 18:03:11.421404 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.421384    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8a48bf8b-77cb998874-r7hvc"]
Jul 30 18:03:11.430832 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.430796    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57"]
Jul 30 18:03:11.434924 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.434896    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8a48bf8b-5ffdcbdb55-9zz57"]
Jul 30 18:03:11.583673 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.583593    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:11.683596 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.683560    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:11.798493 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:11.798434    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:12.111892 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:12.111859    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b32af3a-36a6-4364-89b3-d206a847cf92" path="/var/lib/kubelet/pods/0b32af3a-36a6-4364-89b3-d206a847cf92/volumes"
Jul 30 18:03:12.112324 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:12.112306    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6" path="/var/lib/kubelet/pods/18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6/volumes"
Jul 30 18:03:12.112749 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:12.112724    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a452f4e2-2682-49fe-b96a-2a0d60463409" path="/var/lib/kubelet/pods/a452f4e2-2682-49fe-b96a-2a0d60463409/volumes"
Jul 30 18:03:12.557507 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:12.557463    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:12.621317 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:12.621287    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:14.513229 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:14.513192    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:14.641905 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:14.641875    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj"]
Jul 30 18:03:14.642296 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:14.642282    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a452f4e2-2682-49fe-b96a-2a0d60463409" containerName="payload-processing"
Jul 30 18:03:14.642364 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:14.642299    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a452f4e2-2682-49fe-b96a-2a0d60463409" containerName="payload-processing"
Jul 30 18:03:14.642364 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:14.642310    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b32af3a-36a6-4364-89b3-d206a847cf92" containerName="payload-pre-processing"
Jul 30 18:03:14.642364 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:14.642315    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b32af3a-36a6-4364-89b3-d206a847cf92" containerName="payload-pre-processing"
Jul 30 18:03:14.642364 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:14.642333    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6" containerName="maas-api"
Jul 30 18:03:14.642364 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:14.642338    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6" containerName="maas-api"
Jul 30 18:03:14.642548 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:14.642393    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a452f4e2-2682-49fe-b96a-2a0d60463409" containerName="payload-processing"
Jul 30 18:03:14.642548 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:14.642401    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="18aed62e-ba22-4ab2-99ae-5cf0cb4baaa6" containerName="maas-api"
Jul 30 18:03:14.642548 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:14.642410    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b32af3a-36a6-4364-89b3-d206a847cf92" containerName="payload-pre-processing"
Jul 30 18:03:14.647175 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:14.647158    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj"
Jul 30 18:03:14.649905 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:14.649884    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-fee844af-serving-cert\""
Jul 30 18:03:14.674022 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:14.673997    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj"]
Jul 30 18:03:14.708919 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:14.708878    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:14.722064 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:14.722032    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-maas-api-tls\") pod \"maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj\" (UID: \"9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj"
Jul 30 18:03:14.722219 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:14.722085    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-openshift-service-ca\") pod \"maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj\" (UID: \"9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj"
Jul 30 18:03:14.722219 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:14.722116    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r2ph\" (UniqueName: \"kubernetes.io/projected/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-kube-api-access-7r2ph\") pod \"maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj\" (UID: \"9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj"
Jul 30 18:03:14.823077 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:14.822975    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-maas-api-tls\") pod \"maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj\" (UID: \"9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj"
Jul 30 18:03:14.823077 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:14.823024    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-openshift-service-ca\") pod \"maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj\" (UID: \"9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj"
Jul 30 18:03:14.823077 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:14.823047    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r2ph\" (UniqueName: \"kubernetes.io/projected/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-kube-api-access-7r2ph\") pod \"maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj\" (UID: \"9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj"
Jul 30 18:03:14.823372 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:03:14.823146    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-serving-cert: secret "maas-api-e2e-migrate-fee844af-serving-cert" not found
Jul 30 18:03:14.823372 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:03:14.823262    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-maas-api-tls podName:9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4 nodeName:}" failed. No retries permitted until 2026-07-30 18:03:15.323238375 +0000 UTC m=+2169.776802346 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-maas-api-tls") pod "maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj" (UID: "9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4") : secret "maas-api-e2e-migrate-fee844af-serving-cert" not found
Jul 30 18:03:14.823631 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:14.823613    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-openshift-service-ca\") pod \"maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj\" (UID: \"9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj"
Jul 30 18:03:14.834926 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:14.834902    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r2ph\" (UniqueName: \"kubernetes.io/projected/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-kube-api-access-7r2ph\") pod \"maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj\" (UID: \"9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj"
Jul 30 18:03:15.310319 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.310281    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:15.327717 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.327687    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-maas-api-tls\") pod \"maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj\" (UID: \"9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj"
Jul 30 18:03:15.327923 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:03:15.327870    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-serving-cert: secret "maas-api-e2e-migrate-fee844af-serving-cert" not found
Jul 30 18:03:15.327981 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:03:15.327945    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-maas-api-tls podName:9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4 nodeName:}" failed. No retries permitted until 2026-07-30 18:03:16.327929734 +0000 UTC m=+2170.781493706 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-maas-api-tls") pod "maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj" (UID: "9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4") : secret "maas-api-e2e-migrate-fee844af-serving-cert" not found
Jul 30 18:03:15.412760 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.412726    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:15.425883 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.425851    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt"]
Jul 30 18:03:15.429983 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.429954    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt"
Jul 30 18:03:15.432556 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.432533    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-fee844af-dockercfg-c6hwh\""
Jul 30 18:03:15.432698 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.432574    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-fee844af\""
Jul 30 18:03:15.439043 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.439018    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt"]
Jul 30 18:03:15.448870 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.448844    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw"]
Jul 30 18:03:15.452412 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.452388    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw"
Jul 30 18:03:15.463238 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.463209    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw"]
Jul 30 18:03:15.510977 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.510930    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:15.530362 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.530333    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt\" (UID: \"9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt"
Jul 30 18:03:15.530762 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.530404    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dn6c\" (UniqueName: \"kubernetes.io/projected/b6ca4ea9-7259-4265-85ad-12dd82403f0b-kube-api-access-4dn6c\") pod \"payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw\" (UID: \"b6ca4ea9-7259-4265-85ad-12dd82403f0b\") " pod="openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw"
Jul 30 18:03:15.530762 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.530438    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6ca4ea9-7259-4265-85ad-12dd82403f0b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw\" (UID: \"b6ca4ea9-7259-4265-85ad-12dd82403f0b\") " pod="openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw"
Jul 30 18:03:15.530762 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.530471    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq7m4\" (UniqueName: \"kubernetes.io/projected/9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37-kube-api-access-mq7m4\") pod \"payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt\" (UID: \"9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt"
Jul 30 18:03:15.632169 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.632082    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dn6c\" (UniqueName: \"kubernetes.io/projected/b6ca4ea9-7259-4265-85ad-12dd82403f0b-kube-api-access-4dn6c\") pod \"payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw\" (UID: \"b6ca4ea9-7259-4265-85ad-12dd82403f0b\") " pod="openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw"
Jul 30 18:03:15.632169 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.632122    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6ca4ea9-7259-4265-85ad-12dd82403f0b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw\" (UID: \"b6ca4ea9-7259-4265-85ad-12dd82403f0b\") " pod="openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw"
Jul 30 18:03:15.632169 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.632143    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq7m4\" (UniqueName: \"kubernetes.io/projected/9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37-kube-api-access-mq7m4\") pod \"payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt\" (UID: \"9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt"
Jul 30 18:03:15.632470 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.632238    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt\" (UID: \"9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt"
Jul 30 18:03:15.632948 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.632925    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6ca4ea9-7259-4265-85ad-12dd82403f0b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw\" (UID: \"b6ca4ea9-7259-4265-85ad-12dd82403f0b\") " pod="openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw"
Jul 30 18:03:15.633054 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.632947    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt\" (UID: \"9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt"
Jul 30 18:03:15.640558 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.640532    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dn6c\" (UniqueName: \"kubernetes.io/projected/b6ca4ea9-7259-4265-85ad-12dd82403f0b-kube-api-access-4dn6c\") pod \"payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw\" (UID: \"b6ca4ea9-7259-4265-85ad-12dd82403f0b\") " pod="openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw"
Jul 30 18:03:15.640683 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.640570    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq7m4\" (UniqueName: \"kubernetes.io/projected/9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37-kube-api-access-mq7m4\") pod \"payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt\" (UID: \"9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt"
Jul 30 18:03:15.742539 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.742510    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt"
Jul 30 18:03:15.765858 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.765793    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw"
Jul 30 18:03:15.900240 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.900206    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt"]
Jul 30 18:03:15.902225 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:03:15.902179    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b6fbe1a_6d86_4188_b9c6_c0d798c1ff37.slice/crio-21c299c2e4a0e6ca3a8c0fb0d57c0ba497dad59562a05e379885eb49ddcfc3db WatchSource:0}: Error finding container 21c299c2e4a0e6ca3a8c0fb0d57c0ba497dad59562a05e379885eb49ddcfc3db: Status 404 returned error can't find the container with id 21c299c2e4a0e6ca3a8c0fb0d57c0ba497dad59562a05e379885eb49ddcfc3db
Jul 30 18:03:15.932982 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:15.932957    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw"]
Jul 30 18:03:15.934039 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:03:15.934016    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6ca4ea9_7259_4265_85ad_12dd82403f0b.slice/crio-21ce041e9c83ca11853d451e677c6c338a510019e1916de8dc8745d10d0519eb WatchSource:0}: Error finding container 21ce041e9c83ca11853d451e677c6c338a510019e1916de8dc8745d10d0519eb: Status 404 returned error can't find the container with id 21ce041e9c83ca11853d451e677c6c338a510019e1916de8dc8745d10d0519eb
Jul 30 18:03:16.339608 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:16.339567    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-maas-api-tls\") pod \"maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj\" (UID: \"9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj"
Jul 30 18:03:16.342296 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:16.342270    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-maas-api-tls\") pod \"maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj\" (UID: \"9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj"
Jul 30 18:03:16.409272 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:16.409238    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw" event={"ID":"b6ca4ea9-7259-4265-85ad-12dd82403f0b","Type":"ContainerStarted","Data":"f7dd12a8ecdbd9fb4d650937a14cef7f241e1bc003624741c14e8a7372622a8b"}
Jul 30 18:03:16.409463 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:16.409293    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw" event={"ID":"b6ca4ea9-7259-4265-85ad-12dd82403f0b","Type":"ContainerStarted","Data":"21ce041e9c83ca11853d451e677c6c338a510019e1916de8dc8745d10d0519eb"}
Jul 30 18:03:16.409463 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:16.409348    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw"
Jul 30 18:03:16.411047 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:16.411015    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt" event={"ID":"9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37","Type":"ContainerStarted","Data":"e2d8b3d165267a392749faffde0aaacb7d2bbad799b261368824bd8cc8373307"}
Jul 30 18:03:16.411185 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:16.411065    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt" event={"ID":"9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37","Type":"ContainerStarted","Data":"21c299c2e4a0e6ca3a8c0fb0d57c0ba497dad59562a05e379885eb49ddcfc3db"}
Jul 30 18:03:16.411185 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:16.411092    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt"
Jul 30 18:03:16.426383 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:16.426326    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw" podStartSLOduration=1.426312525 podStartE2EDuration="1.426312525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:03:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:03:16.423854148 +0000 UTC m=+2170.877418138" watchObservedRunningTime="2026-07-30 18:03:16.426312525 +0000 UTC m=+2170.879876506"
Jul 30 18:03:16.439922 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:16.439872    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt" podStartSLOduration=1.439855897 podStartE2EDuration="1.439855897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:03:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:03:16.438204496 +0000 UTC m=+2170.891768487" watchObservedRunningTime="2026-07-30 18:03:16.439855897 +0000 UTC m=+2170.893419888"
Jul 30 18:03:16.458967 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:16.458925    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj"
Jul 30 18:03:16.616689 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:16.616659    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj"]
Jul 30 18:03:16.618096 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:03:16.618068    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d2ba665_4cdd_425c_9424_4c1e9d2ddbe4.slice/crio-7dafa701baea4828c5d04d640a29969b0eea4e849a05be79da0b03d2ee75345e WatchSource:0}: Error finding container 7dafa701baea4828c5d04d640a29969b0eea4e849a05be79da0b03d2ee75345e: Status 404 returned error can't find the container with id 7dafa701baea4828c5d04d640a29969b0eea4e849a05be79da0b03d2ee75345e
Jul 30 18:03:17.417556 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:17.417518    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj" event={"ID":"9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4","Type":"ContainerStarted","Data":"7dafa701baea4828c5d04d640a29969b0eea4e849a05be79da0b03d2ee75345e"}
Jul 30 18:03:18.424332 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:18.424297    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj" event={"ID":"9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4","Type":"ContainerStarted","Data":"6f75cd20375a5434975c61edb404d13f406783d926658d773b3d6bf91947d5e9"}
Jul 30 18:03:18.424702 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:18.424402    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj"
Jul 30 18:03:18.442152 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:18.442102    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj" podStartSLOduration=3.433415304 podStartE2EDuration="4.442086853s" totalImagesPullingTime="1.008671549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:03:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:03:18.44074439 +0000 UTC m=+2172.894308383" watchObservedRunningTime="2026-07-30 18:03:18.442086853 +0000 UTC m=+2172.895650844"
Jul 30 18:03:24.436209 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:24.436181    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj"
Jul 30 18:03:27.420198 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:27.420165    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw"
Jul 30 18:03:27.420634 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:27.420258    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt"
Jul 30 18:03:45.894118 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:45.894079    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj"]
Jul 30 18:03:45.894611 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:45.894373    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj" podUID="9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4" containerName="maas-api" containerID="cri-o://6f75cd20375a5434975c61edb404d13f406783d926658d773b3d6bf91947d5e9" gracePeriod=30
Jul 30 18:03:45.918075 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:45.918009    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:45.950439 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:45.950406    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw"]
Jul 30 18:03:45.950679 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:45.950633    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw" podUID="b6ca4ea9-7259-4265-85ad-12dd82403f0b" containerName="payload-processing" containerID="cri-o://f7dd12a8ecdbd9fb4d650937a14cef7f241e1bc003624741c14e8a7372622a8b" gracePeriod=30
Jul 30 18:03:45.964950 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:45.964918    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt"]
Jul 30 18:03:45.965185 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:45.965141    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt" podUID="9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37" containerName="payload-pre-processing" containerID="cri-o://e2d8b3d165267a392749faffde0aaacb7d2bbad799b261368824bd8cc8373307" gracePeriod=30
Jul 30 18:03:46.008598 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.008565    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:46.081651 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.081615    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:46.148810 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.148735    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:46.248532 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.248471    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw"
Jul 30 18:03:46.278526 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.278502    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj"
Jul 30 18:03:46.287587 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.287560    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt"
Jul 30 18:03:46.312061 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.312031    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:46.326158 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.326127    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6ca4ea9-7259-4265-85ad-12dd82403f0b-plugins-config-volume\") pod \"b6ca4ea9-7259-4265-85ad-12dd82403f0b\" (UID: \"b6ca4ea9-7259-4265-85ad-12dd82403f0b\") "
Jul 30 18:03:46.326334 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.326170    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dn6c\" (UniqueName: \"kubernetes.io/projected/b6ca4ea9-7259-4265-85ad-12dd82403f0b-kube-api-access-4dn6c\") pod \"b6ca4ea9-7259-4265-85ad-12dd82403f0b\" (UID: \"b6ca4ea9-7259-4265-85ad-12dd82403f0b\") "
Jul 30 18:03:46.326334 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.326202    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-openshift-service-ca\") pod \"9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4\" (UID: \"9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4\") "
Jul 30 18:03:46.326334 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.326253    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-maas-api-tls\") pod \"9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4\" (UID: \"9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4\") "
Jul 30 18:03:46.326334 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.326313    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7r2ph\" (UniqueName: \"kubernetes.io/projected/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-kube-api-access-7r2ph\") pod \"9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4\" (UID: \"9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4\") "
Jul 30 18:03:46.326557 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.326476    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6ca4ea9-7259-4265-85ad-12dd82403f0b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b6ca4ea9-7259-4265-85ad-12dd82403f0b" (UID: "b6ca4ea9-7259-4265-85ad-12dd82403f0b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:03:46.326610 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.326580    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4" (UID: "9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:03:46.326665 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.326619    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6ca4ea9-7259-4265-85ad-12dd82403f0b-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:03:46.328564 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.328539    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-kube-api-access-7r2ph" (OuterVolumeSpecName: "kube-api-access-7r2ph") pod "9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4" (UID: "9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4"). InnerVolumeSpecName "kube-api-access-7r2ph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:03:46.328683 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.328588    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4" (UID: "9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 18:03:46.328740 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.328726    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6ca4ea9-7259-4265-85ad-12dd82403f0b-kube-api-access-4dn6c" (OuterVolumeSpecName: "kube-api-access-4dn6c") pod "b6ca4ea9-7259-4265-85ad-12dd82403f0b" (UID: "b6ca4ea9-7259-4265-85ad-12dd82403f0b"). InnerVolumeSpecName "kube-api-access-4dn6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:03:46.427121 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.427043    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mq7m4\" (UniqueName: \"kubernetes.io/projected/9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37-kube-api-access-mq7m4\") pod \"9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37\" (UID: \"9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37\") "
Jul 30 18:03:46.427121 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.427082    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37-plugins-config-volume\") pod \"9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37\" (UID: \"9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37\") "
Jul 30 18:03:46.427433 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.427416    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dn6c\" (UniqueName: \"kubernetes.io/projected/b6ca4ea9-7259-4265-85ad-12dd82403f0b-kube-api-access-4dn6c\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:03:46.427483 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.427440    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-openshift-service-ca\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:03:46.427483 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.427454    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-maas-api-tls\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:03:46.427483 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.427467    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7r2ph\" (UniqueName: \"kubernetes.io/projected/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4-kube-api-access-7r2ph\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:03:46.427588 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.427478    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37" (UID: "9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:03:46.429362 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.429341    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37-kube-api-access-mq7m4" (OuterVolumeSpecName: "kube-api-access-mq7m4") pod "9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37" (UID: "9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37"). InnerVolumeSpecName "kube-api-access-mq7m4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:03:46.528195 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.528156    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mq7m4\" (UniqueName: \"kubernetes.io/projected/9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37-kube-api-access-mq7m4\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:03:46.528195 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.528182    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:03:46.564323 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.564276    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:46.585390 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.585356    2577 generic.go:356] "Generic (PLEG): container finished" podID="b6ca4ea9-7259-4265-85ad-12dd82403f0b" containerID="f7dd12a8ecdbd9fb4d650937a14cef7f241e1bc003624741c14e8a7372622a8b" exitCode=0
Jul 30 18:03:46.585545 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.585405    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw"
Jul 30 18:03:46.585545 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.585436    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw" event={"ID":"b6ca4ea9-7259-4265-85ad-12dd82403f0b","Type":"ContainerDied","Data":"f7dd12a8ecdbd9fb4d650937a14cef7f241e1bc003624741c14e8a7372622a8b"}
Jul 30 18:03:46.585545 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.585505    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw" event={"ID":"b6ca4ea9-7259-4265-85ad-12dd82403f0b","Type":"ContainerDied","Data":"21ce041e9c83ca11853d451e677c6c338a510019e1916de8dc8745d10d0519eb"}
Jul 30 18:03:46.585545 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.585539    2577 scope.go:117] "RemoveContainer" containerID="f7dd12a8ecdbd9fb4d650937a14cef7f241e1bc003624741c14e8a7372622a8b"
Jul 30 18:03:46.586782 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.586758    2577 generic.go:356] "Generic (PLEG): container finished" podID="9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37" containerID="e2d8b3d165267a392749faffde0aaacb7d2bbad799b261368824bd8cc8373307" exitCode=0
Jul 30 18:03:46.586932 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.586808    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt"
Jul 30 18:03:46.586932 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.586843    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt" event={"ID":"9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37","Type":"ContainerDied","Data":"e2d8b3d165267a392749faffde0aaacb7d2bbad799b261368824bd8cc8373307"}
Jul 30 18:03:46.586932 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.586896    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt" event={"ID":"9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37","Type":"ContainerDied","Data":"21c299c2e4a0e6ca3a8c0fb0d57c0ba497dad59562a05e379885eb49ddcfc3db"}
Jul 30 18:03:46.588438 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.588418    2577 generic.go:356] "Generic (PLEG): container finished" podID="9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4" containerID="6f75cd20375a5434975c61edb404d13f406783d926658d773b3d6bf91947d5e9" exitCode=0
Jul 30 18:03:46.588559 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.588438    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj" event={"ID":"9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4","Type":"ContainerDied","Data":"6f75cd20375a5434975c61edb404d13f406783d926658d773b3d6bf91947d5e9"}
Jul 30 18:03:46.588559 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.588458    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj"
Jul 30 18:03:46.588559 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.588484    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj" event={"ID":"9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4","Type":"ContainerDied","Data":"7dafa701baea4828c5d04d640a29969b0eea4e849a05be79da0b03d2ee75345e"}
Jul 30 18:03:46.595165 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.595005    2577 scope.go:117] "RemoveContainer" containerID="f7dd12a8ecdbd9fb4d650937a14cef7f241e1bc003624741c14e8a7372622a8b"
Jul 30 18:03:46.595415 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:03:46.595391    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7dd12a8ecdbd9fb4d650937a14cef7f241e1bc003624741c14e8a7372622a8b\": container with ID starting with f7dd12a8ecdbd9fb4d650937a14cef7f241e1bc003624741c14e8a7372622a8b not found: ID does not exist" containerID="f7dd12a8ecdbd9fb4d650937a14cef7f241e1bc003624741c14e8a7372622a8b"
Jul 30 18:03:46.595513 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.595428    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7dd12a8ecdbd9fb4d650937a14cef7f241e1bc003624741c14e8a7372622a8b"} err="failed to get container status \"f7dd12a8ecdbd9fb4d650937a14cef7f241e1bc003624741c14e8a7372622a8b\": rpc error: code = NotFound desc = could not find container \"f7dd12a8ecdbd9fb4d650937a14cef7f241e1bc003624741c14e8a7372622a8b\": container with ID starting with f7dd12a8ecdbd9fb4d650937a14cef7f241e1bc003624741c14e8a7372622a8b not found: ID does not exist"
Jul 30 18:03:46.595513 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.595452    2577 scope.go:117] "RemoveContainer" containerID="e2d8b3d165267a392749faffde0aaacb7d2bbad799b261368824bd8cc8373307"
Jul 30 18:03:46.603915 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.603894    2577 scope.go:117] "RemoveContainer" containerID="e2d8b3d165267a392749faffde0aaacb7d2bbad799b261368824bd8cc8373307"
Jul 30 18:03:46.604169 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:03:46.604153    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2d8b3d165267a392749faffde0aaacb7d2bbad799b261368824bd8cc8373307\": container with ID starting with e2d8b3d165267a392749faffde0aaacb7d2bbad799b261368824bd8cc8373307 not found: ID does not exist" containerID="e2d8b3d165267a392749faffde0aaacb7d2bbad799b261368824bd8cc8373307"
Jul 30 18:03:46.604225 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.604176    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2d8b3d165267a392749faffde0aaacb7d2bbad799b261368824bd8cc8373307"} err="failed to get container status \"e2d8b3d165267a392749faffde0aaacb7d2bbad799b261368824bd8cc8373307\": rpc error: code = NotFound desc = could not find container \"e2d8b3d165267a392749faffde0aaacb7d2bbad799b261368824bd8cc8373307\": container with ID starting with e2d8b3d165267a392749faffde0aaacb7d2bbad799b261368824bd8cc8373307 not found: ID does not exist"
Jul 30 18:03:46.604225 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.604190    2577 scope.go:117] "RemoveContainer" containerID="6f75cd20375a5434975c61edb404d13f406783d926658d773b3d6bf91947d5e9"
Jul 30 18:03:46.612298 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.612281    2577 scope.go:117] "RemoveContainer" containerID="6f75cd20375a5434975c61edb404d13f406783d926658d773b3d6bf91947d5e9"
Jul 30 18:03:46.612550 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:03:46.612534    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f75cd20375a5434975c61edb404d13f406783d926658d773b3d6bf91947d5e9\": container with ID starting with 6f75cd20375a5434975c61edb404d13f406783d926658d773b3d6bf91947d5e9 not found: ID does not exist" containerID="6f75cd20375a5434975c61edb404d13f406783d926658d773b3d6bf91947d5e9"
Jul 30 18:03:46.612601 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.612558    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f75cd20375a5434975c61edb404d13f406783d926658d773b3d6bf91947d5e9"} err="failed to get container status \"6f75cd20375a5434975c61edb404d13f406783d926658d773b3d6bf91947d5e9\": rpc error: code = NotFound desc = could not find container \"6f75cd20375a5434975c61edb404d13f406783d926658d773b3d6bf91947d5e9\": container with ID starting with 6f75cd20375a5434975c61edb404d13f406783d926658d773b3d6bf91947d5e9 not found: ID does not exist"
Jul 30 18:03:46.620308 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.620287    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj"]
Jul 30 18:03:46.624698 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.624678    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fee844af-698685ccfd-jd2rj"]
Jul 30 18:03:46.635292 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.635273    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw"]
Jul 30 18:03:46.639571 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.639552    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fee844af-6ff7f7df4f-xh6gw"]
Jul 30 18:03:46.652031 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.652009    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt"]
Jul 30 18:03:46.656253 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.656233    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fee844af-cd897776d-8kbtt"]
Jul 30 18:03:46.762452 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:46.762421    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:47.193461 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:47.193360    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:47.293233 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:47.293190    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:48.111006 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:48.110953    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37" path="/var/lib/kubelet/pods/9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37/volumes"
Jul 30 18:03:48.111349 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:48.111335    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4" path="/var/lib/kubelet/pods/9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4/volumes"
Jul 30 18:03:48.111639 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:48.111628    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6ca4ea9-7259-4265-85ad-12dd82403f0b" path="/var/lib/kubelet/pods/b6ca4ea9-7259-4265-85ad-12dd82403f0b/volumes"
Jul 30 18:03:49.509356 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:49.509322    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:49.603688 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:49.603654    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:50.631337 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.631306    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs"]
Jul 30 18:03:50.631742 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.631727    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37" containerName="payload-pre-processing"
Jul 30 18:03:50.631786 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.631745    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37" containerName="payload-pre-processing"
Jul 30 18:03:50.631786 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.631768    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6ca4ea9-7259-4265-85ad-12dd82403f0b" containerName="payload-processing"
Jul 30 18:03:50.631786 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.631774    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6ca4ea9-7259-4265-85ad-12dd82403f0b" containerName="payload-processing"
Jul 30 18:03:50.631906 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.631791    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4" containerName="maas-api"
Jul 30 18:03:50.631906 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.631797    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4" containerName="maas-api"
Jul 30 18:03:50.631906 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.631867    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b6fbe1a-6d86-4188-b9c6-c0d798c1ff37" containerName="payload-pre-processing"
Jul 30 18:03:50.631906 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.631876    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d2ba665-4cdd-425c-9424-4c1e9d2ddbe4" containerName="maas-api"
Jul 30 18:03:50.631906 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.631886    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6ca4ea9-7259-4265-85ad-12dd82403f0b" containerName="payload-processing"
Jul 30 18:03:50.636558 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.636539    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs"
Jul 30 18:03:50.639044 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.639009    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-c2052f19-serving-cert\""
Jul 30 18:03:50.647265 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.647244    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs"]
Jul 30 18:03:50.688732 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.688701    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:50.771869 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.771811    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/079ca29d-267e-4cc1-80bc-e241193399b8-openshift-service-ca\") pod \"maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs\" (UID: \"079ca29d-267e-4cc1-80bc-e241193399b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs"
Jul 30 18:03:50.771869 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.771865    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t2qk\" (UniqueName: \"kubernetes.io/projected/079ca29d-267e-4cc1-80bc-e241193399b8-kube-api-access-7t2qk\") pod \"maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs\" (UID: \"079ca29d-267e-4cc1-80bc-e241193399b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs"
Jul 30 18:03:50.772075 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.771953    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/079ca29d-267e-4cc1-80bc-e241193399b8-maas-api-tls\") pod \"maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs\" (UID: \"079ca29d-267e-4cc1-80bc-e241193399b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs"
Jul 30 18:03:50.806441 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.806415    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:50.873253 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.873219    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/079ca29d-267e-4cc1-80bc-e241193399b8-maas-api-tls\") pod \"maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs\" (UID: \"079ca29d-267e-4cc1-80bc-e241193399b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs"
Jul 30 18:03:50.873414 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.873325    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/079ca29d-267e-4cc1-80bc-e241193399b8-openshift-service-ca\") pod \"maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs\" (UID: \"079ca29d-267e-4cc1-80bc-e241193399b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs"
Jul 30 18:03:50.873414 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.873345    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t2qk\" (UniqueName: \"kubernetes.io/projected/079ca29d-267e-4cc1-80bc-e241193399b8-kube-api-access-7t2qk\") pod \"maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs\" (UID: \"079ca29d-267e-4cc1-80bc-e241193399b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs"
Jul 30 18:03:50.874071 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.873969    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/079ca29d-267e-4cc1-80bc-e241193399b8-openshift-service-ca\") pod \"maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs\" (UID: \"079ca29d-267e-4cc1-80bc-e241193399b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs"
Jul 30 18:03:50.875778 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.875754    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/079ca29d-267e-4cc1-80bc-e241193399b8-maas-api-tls\") pod \"maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs\" (UID: \"079ca29d-267e-4cc1-80bc-e241193399b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs"
Jul 30 18:03:50.883642 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.883585    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t2qk\" (UniqueName: \"kubernetes.io/projected/079ca29d-267e-4cc1-80bc-e241193399b8-kube-api-access-7t2qk\") pod \"maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs\" (UID: \"079ca29d-267e-4cc1-80bc-e241193399b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs"
Jul 30 18:03:50.947625 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:50.947594    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs"
Jul 30 18:03:51.094433 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.094396    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs"]
Jul 30 18:03:51.096850 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:03:51.096791    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod079ca29d_267e_4cc1_80bc_e241193399b8.slice/crio-828f59122aa1b57f7adb66f2fa32746d3ba019e0da76ee72bd73666c4c0445c6 WatchSource:0}: Error finding container 828f59122aa1b57f7adb66f2fa32746d3ba019e0da76ee72bd73666c4c0445c6: Status 404 returned error can't find the container with id 828f59122aa1b57f7adb66f2fa32746d3ba019e0da76ee72bd73666c4c0445c6
Jul 30 18:03:51.342923 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.342895    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v"]
Jul 30 18:03:51.347576 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.347560    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v"
Jul 30 18:03:51.349897 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.349880    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-c2052f19\""
Jul 30 18:03:51.358652 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.358628    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v"]
Jul 30 18:03:51.366516 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.366488    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c"]
Jul 30 18:03:51.372211 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.372190    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c"
Jul 30 18:03:51.374561 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.374536    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-c2052f19-dockercfg-s6kl2\""
Jul 30 18:03:51.383408 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.383389    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c"]
Jul 30 18:03:51.419359 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.419332    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:51.479556 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.479523    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbcdf\" (UniqueName: \"kubernetes.io/projected/a895919c-c08e-45ee-860b-49fb56806b32-kube-api-access-jbcdf\") pod \"payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c\" (UID: \"a895919c-c08e-45ee-860b-49fb56806b32\") " pod="openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c"
Jul 30 18:03:51.479703 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.479580    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ed7543-df6d-4265-bfbc-655a62928418-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v\" (UID: \"69ed7543-df6d-4265-bfbc-655a62928418\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v"
Jul 30 18:03:51.479703 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.479623    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp28h\" (UniqueName: \"kubernetes.io/projected/69ed7543-df6d-4265-bfbc-655a62928418-kube-api-access-wp28h\") pod \"payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v\" (UID: \"69ed7543-df6d-4265-bfbc-655a62928418\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v"
Jul 30 18:03:51.479703 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.479684    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a895919c-c08e-45ee-860b-49fb56806b32-plugins-config-volume\") pod \"payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c\" (UID: \"a895919c-c08e-45ee-860b-49fb56806b32\") " pod="openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c"
Jul 30 18:03:51.520180 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.520149    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:03:51.580795 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.580757    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbcdf\" (UniqueName: \"kubernetes.io/projected/a895919c-c08e-45ee-860b-49fb56806b32-kube-api-access-jbcdf\") pod \"payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c\" (UID: \"a895919c-c08e-45ee-860b-49fb56806b32\") " pod="openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c"
Jul 30 18:03:51.580973 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.580810    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ed7543-df6d-4265-bfbc-655a62928418-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v\" (UID: \"69ed7543-df6d-4265-bfbc-655a62928418\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v"
Jul 30 18:03:51.580973 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.580952    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp28h\" (UniqueName: \"kubernetes.io/projected/69ed7543-df6d-4265-bfbc-655a62928418-kube-api-access-wp28h\") pod \"payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v\" (UID: \"69ed7543-df6d-4265-bfbc-655a62928418\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v"
Jul 30 18:03:51.581100 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.581083    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a895919c-c08e-45ee-860b-49fb56806b32-plugins-config-volume\") pod \"payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c\" (UID: \"a895919c-c08e-45ee-860b-49fb56806b32\") " pod="openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c"
Jul 30 18:03:51.581666 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.581641    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ed7543-df6d-4265-bfbc-655a62928418-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v\" (UID: \"69ed7543-df6d-4265-bfbc-655a62928418\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v"
Jul 30 18:03:51.581901 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.581883    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a895919c-c08e-45ee-860b-49fb56806b32-plugins-config-volume\") pod \"payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c\" (UID: \"a895919c-c08e-45ee-860b-49fb56806b32\") " pod="openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c"
Jul 30 18:03:51.589116 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.589096    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbcdf\" (UniqueName: \"kubernetes.io/projected/a895919c-c08e-45ee-860b-49fb56806b32-kube-api-access-jbcdf\") pod \"payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c\" (UID: \"a895919c-c08e-45ee-860b-49fb56806b32\") " pod="openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c"
Jul 30 18:03:51.589233 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.589212    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp28h\" (UniqueName: \"kubernetes.io/projected/69ed7543-df6d-4265-bfbc-655a62928418-kube-api-access-wp28h\") pod \"payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v\" (UID: \"69ed7543-df6d-4265-bfbc-655a62928418\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v"
Jul 30 18:03:51.621765 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.621696    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs" event={"ID":"079ca29d-267e-4cc1-80bc-e241193399b8","Type":"ContainerStarted","Data":"828f59122aa1b57f7adb66f2fa32746d3ba019e0da76ee72bd73666c4c0445c6"}
Jul 30 18:03:51.660775 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.660747    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v"
Jul 30 18:03:51.682482 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.682433    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c"
Jul 30 18:03:51.822482 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.822453    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v"]
Jul 30 18:03:51.823282 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:03:51.822893    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69ed7543_df6d_4265_bfbc_655a62928418.slice/crio-858ef79246d6c315fcc5262674fd4ea4b11b5a23454f22be2760eb3161ef1e50 WatchSource:0}: Error finding container 858ef79246d6c315fcc5262674fd4ea4b11b5a23454f22be2760eb3161ef1e50: Status 404 returned error can't find the container with id 858ef79246d6c315fcc5262674fd4ea4b11b5a23454f22be2760eb3161ef1e50
Jul 30 18:03:51.850593 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:51.850562    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c"]
Jul 30 18:03:52.631327 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:52.630083    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs" event={"ID":"079ca29d-267e-4cc1-80bc-e241193399b8","Type":"ContainerStarted","Data":"c134ef682b052f8aa531f722fc7c1bb6b945af2d43c01750356c971be576c1ca"}
Jul 30 18:03:52.631327 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:52.631189    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs"
Jul 30 18:03:52.633423 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:52.633390    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v" event={"ID":"69ed7543-df6d-4265-bfbc-655a62928418","Type":"ContainerStarted","Data":"467b61401d65c3d3cdc0f1a30801ad160061eeb907e9cb32955e42152f5cf7b9"}
Jul 30 18:03:52.633574 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:52.633442    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v" event={"ID":"69ed7543-df6d-4265-bfbc-655a62928418","Type":"ContainerStarted","Data":"858ef79246d6c315fcc5262674fd4ea4b11b5a23454f22be2760eb3161ef1e50"}
Jul 30 18:03:52.633574 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:52.633468    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v"
Jul 30 18:03:52.635009 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:52.634967    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c" event={"ID":"a895919c-c08e-45ee-860b-49fb56806b32","Type":"ContainerStarted","Data":"51ebc9df2674bbc36b9af9b61605fbbaeab79d38100e92b6d4e91d77677a66d1"}
Jul 30 18:03:52.635149 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:52.635023    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c" event={"ID":"a895919c-c08e-45ee-860b-49fb56806b32","Type":"ContainerStarted","Data":"a3f6fcd6bb81f4593fd1d847b157656ff0f6c468de6a7baa562b520f378c6f6b"}
Jul 30 18:03:52.635149 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:52.635114    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c"
Jul 30 18:03:52.648863 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:52.648811    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs" podStartSLOduration=1.696697054 podStartE2EDuration="2.648797669s" totalImagesPullingTime="952.100615ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:03:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:03:52.646455473 +0000 UTC m=+2207.100019463" watchObservedRunningTime="2026-07-30 18:03:52.648797669 +0000 UTC m=+2207.102361658"
Jul 30 18:03:52.663417 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:52.663370    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v" podStartSLOduration=1.663355793 podStartE2EDuration="1.663355793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:03:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:03:52.661282445 +0000 UTC m=+2207.114846436" watchObservedRunningTime="2026-07-30 18:03:52.663355793 +0000 UTC m=+2207.116919784"
Jul 30 18:03:52.679521 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:52.679477    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c" podStartSLOduration=1.679465021 podStartE2EDuration="1.679465021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:03:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:03:52.678046745 +0000 UTC m=+2207.131610735" watchObservedRunningTime="2026-07-30 18:03:52.679465021 +0000 UTC m=+2207.133029012"
Jul 30 18:03:59.651950 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:03:59.651917    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs"
Jul 30 18:04:03.642017 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:03.641988    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c"
Jul 30 18:04:03.642405 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:03.642149    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v"
Jul 30 18:04:21.654571 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:21.654527    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs"]
Jul 30 18:04:21.655074 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:21.654849    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs" podUID="079ca29d-267e-4cc1-80bc-e241193399b8" containerName="maas-api" containerID="cri-o://c134ef682b052f8aa531f722fc7c1bb6b945af2d43c01750356c971be576c1ca" gracePeriod=30
Jul 30 18:04:21.677416 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:21.677366    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:04:21.708767 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:21.708734    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c"]
Jul 30 18:04:21.709156 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:21.709071    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c" podUID="a895919c-c08e-45ee-860b-49fb56806b32" containerName="payload-processing" containerID="cri-o://51ebc9df2674bbc36b9af9b61605fbbaeab79d38100e92b6d4e91d77677a66d1" gracePeriod=30
Jul 30 18:04:21.716517 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:21.716490    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v"]
Jul 30 18:04:21.716924 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:21.716894    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v" podUID="69ed7543-df6d-4265-bfbc-655a62928418" containerName="payload-pre-processing" containerID="cri-o://467b61401d65c3d3cdc0f1a30801ad160061eeb907e9cb32955e42152f5cf7b9" gracePeriod=30
Jul 30 18:04:21.771103 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:21.771070    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:04:21.794727 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:21.794675    2577 generic.go:356] "Generic (PLEG): container finished" podID="079ca29d-267e-4cc1-80bc-e241193399b8" containerID="c134ef682b052f8aa531f722fc7c1bb6b945af2d43c01750356c971be576c1ca" exitCode=0
Jul 30 18:04:21.794906 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:21.794758    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs" event={"ID":"079ca29d-267e-4cc1-80bc-e241193399b8","Type":"ContainerDied","Data":"c134ef682b052f8aa531f722fc7c1bb6b945af2d43c01750356c971be576c1ca"}
Jul 30 18:04:21.872345 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:21.872279    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:04:21.975770 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:21.975684    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:04:22.006975 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.006945    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c"
Jul 30 18:04:22.022314 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.022294    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs"
Jul 30 18:04:22.029110 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.029086    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v"
Jul 30 18:04:22.067359 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.067290    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/079ca29d-267e-4cc1-80bc-e241193399b8-openshift-service-ca\") pod \"079ca29d-267e-4cc1-80bc-e241193399b8\" (UID: \"079ca29d-267e-4cc1-80bc-e241193399b8\") "
Jul 30 18:04:22.067359 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.067336    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ed7543-df6d-4265-bfbc-655a62928418-plugins-config-volume\") pod \"69ed7543-df6d-4265-bfbc-655a62928418\" (UID: \"69ed7543-df6d-4265-bfbc-655a62928418\") "
Jul 30 18:04:22.067600 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.067516    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wp28h\" (UniqueName: \"kubernetes.io/projected/69ed7543-df6d-4265-bfbc-655a62928418-kube-api-access-wp28h\") pod \"69ed7543-df6d-4265-bfbc-655a62928418\" (UID: \"69ed7543-df6d-4265-bfbc-655a62928418\") "
Jul 30 18:04:22.067600 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.067564    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbcdf\" (UniqueName: \"kubernetes.io/projected/a895919c-c08e-45ee-860b-49fb56806b32-kube-api-access-jbcdf\") pod \"a895919c-c08e-45ee-860b-49fb56806b32\" (UID: \"a895919c-c08e-45ee-860b-49fb56806b32\") "
Jul 30 18:04:22.067710 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.067622    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7t2qk\" (UniqueName: \"kubernetes.io/projected/079ca29d-267e-4cc1-80bc-e241193399b8-kube-api-access-7t2qk\") pod \"079ca29d-267e-4cc1-80bc-e241193399b8\" (UID: \"079ca29d-267e-4cc1-80bc-e241193399b8\") "
Jul 30 18:04:22.067710 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.067661    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/079ca29d-267e-4cc1-80bc-e241193399b8-maas-api-tls\") pod \"079ca29d-267e-4cc1-80bc-e241193399b8\" (UID: \"079ca29d-267e-4cc1-80bc-e241193399b8\") "
Jul 30 18:04:22.067853 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.067726    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a895919c-c08e-45ee-860b-49fb56806b32-plugins-config-volume\") pod \"a895919c-c08e-45ee-860b-49fb56806b32\" (UID: \"a895919c-c08e-45ee-860b-49fb56806b32\") "
Jul 30 18:04:22.068310 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.067723    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/079ca29d-267e-4cc1-80bc-e241193399b8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "079ca29d-267e-4cc1-80bc-e241193399b8" (UID: "079ca29d-267e-4cc1-80bc-e241193399b8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:04:22.068310 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.067767    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69ed7543-df6d-4265-bfbc-655a62928418-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "69ed7543-df6d-4265-bfbc-655a62928418" (UID: "69ed7543-df6d-4265-bfbc-655a62928418"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:04:22.068874 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.068834    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a895919c-c08e-45ee-860b-49fb56806b32-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a895919c-c08e-45ee-860b-49fb56806b32" (UID: "a895919c-c08e-45ee-860b-49fb56806b32"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:04:22.070197 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.070167    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/079ca29d-267e-4cc1-80bc-e241193399b8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "079ca29d-267e-4cc1-80bc-e241193399b8" (UID: "079ca29d-267e-4cc1-80bc-e241193399b8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 18:04:22.070364 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.070342    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/079ca29d-267e-4cc1-80bc-e241193399b8-kube-api-access-7t2qk" (OuterVolumeSpecName: "kube-api-access-7t2qk") pod "079ca29d-267e-4cc1-80bc-e241193399b8" (UID: "079ca29d-267e-4cc1-80bc-e241193399b8"). InnerVolumeSpecName "kube-api-access-7t2qk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:04:22.070501 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.070482    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a895919c-c08e-45ee-860b-49fb56806b32-kube-api-access-jbcdf" (OuterVolumeSpecName: "kube-api-access-jbcdf") pod "a895919c-c08e-45ee-860b-49fb56806b32" (UID: "a895919c-c08e-45ee-860b-49fb56806b32"). InnerVolumeSpecName "kube-api-access-jbcdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:04:22.070547 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.070500    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69ed7543-df6d-4265-bfbc-655a62928418-kube-api-access-wp28h" (OuterVolumeSpecName: "kube-api-access-wp28h") pod "69ed7543-df6d-4265-bfbc-655a62928418" (UID: "69ed7543-df6d-4265-bfbc-655a62928418"). InnerVolumeSpecName "kube-api-access-wp28h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:04:22.169291 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.169257    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/079ca29d-267e-4cc1-80bc-e241193399b8-openshift-service-ca\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:04:22.169491 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.169463    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ed7543-df6d-4265-bfbc-655a62928418-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:04:22.169563 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.169494    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wp28h\" (UniqueName: \"kubernetes.io/projected/69ed7543-df6d-4265-bfbc-655a62928418-kube-api-access-wp28h\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:04:22.169563 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.169511    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbcdf\" (UniqueName: \"kubernetes.io/projected/a895919c-c08e-45ee-860b-49fb56806b32-kube-api-access-jbcdf\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:04:22.169563 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.169526    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7t2qk\" (UniqueName: \"kubernetes.io/projected/079ca29d-267e-4cc1-80bc-e241193399b8-kube-api-access-7t2qk\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:04:22.169563 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.169546    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/079ca29d-267e-4cc1-80bc-e241193399b8-maas-api-tls\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:04:22.169563 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.169563    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a895919c-c08e-45ee-860b-49fb56806b32-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:04:22.801024 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.800989    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs" event={"ID":"079ca29d-267e-4cc1-80bc-e241193399b8","Type":"ContainerDied","Data":"828f59122aa1b57f7adb66f2fa32746d3ba019e0da76ee72bd73666c4c0445c6"}
Jul 30 18:04:22.801486 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.801056    2577 scope.go:117] "RemoveContainer" containerID="c134ef682b052f8aa531f722fc7c1bb6b945af2d43c01750356c971be576c1ca"
Jul 30 18:04:22.801486 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.800994    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs"
Jul 30 18:04:22.802399 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.802327    2577 generic.go:356] "Generic (PLEG): container finished" podID="69ed7543-df6d-4265-bfbc-655a62928418" containerID="467b61401d65c3d3cdc0f1a30801ad160061eeb907e9cb32955e42152f5cf7b9" exitCode=0
Jul 30 18:04:22.802399 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.802366    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v"
Jul 30 18:04:22.802520 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.802358    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v" event={"ID":"69ed7543-df6d-4265-bfbc-655a62928418","Type":"ContainerDied","Data":"467b61401d65c3d3cdc0f1a30801ad160061eeb907e9cb32955e42152f5cf7b9"}
Jul 30 18:04:22.802520 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.802499    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v" event={"ID":"69ed7543-df6d-4265-bfbc-655a62928418","Type":"ContainerDied","Data":"858ef79246d6c315fcc5262674fd4ea4b11b5a23454f22be2760eb3161ef1e50"}
Jul 30 18:04:22.804120 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.804094    2577 generic.go:356] "Generic (PLEG): container finished" podID="a895919c-c08e-45ee-860b-49fb56806b32" containerID="51ebc9df2674bbc36b9af9b61605fbbaeab79d38100e92b6d4e91d77677a66d1" exitCode=0
Jul 30 18:04:22.804241 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.804132    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c"
Jul 30 18:04:22.804241 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.804160    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c" event={"ID":"a895919c-c08e-45ee-860b-49fb56806b32","Type":"ContainerDied","Data":"51ebc9df2674bbc36b9af9b61605fbbaeab79d38100e92b6d4e91d77677a66d1"}
Jul 30 18:04:22.804241 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.804201    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c" event={"ID":"a895919c-c08e-45ee-860b-49fb56806b32","Type":"ContainerDied","Data":"a3f6fcd6bb81f4593fd1d847b157656ff0f6c468de6a7baa562b520f378c6f6b"}
Jul 30 18:04:22.812226 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.812211    2577 scope.go:117] "RemoveContainer" containerID="467b61401d65c3d3cdc0f1a30801ad160061eeb907e9cb32955e42152f5cf7b9"
Jul 30 18:04:22.820635 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.820558    2577 scope.go:117] "RemoveContainer" containerID="467b61401d65c3d3cdc0f1a30801ad160061eeb907e9cb32955e42152f5cf7b9"
Jul 30 18:04:22.820889 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:04:22.820866    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"467b61401d65c3d3cdc0f1a30801ad160061eeb907e9cb32955e42152f5cf7b9\": container with ID starting with 467b61401d65c3d3cdc0f1a30801ad160061eeb907e9cb32955e42152f5cf7b9 not found: ID does not exist" containerID="467b61401d65c3d3cdc0f1a30801ad160061eeb907e9cb32955e42152f5cf7b9"
Jul 30 18:04:22.820949 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.820902    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"467b61401d65c3d3cdc0f1a30801ad160061eeb907e9cb32955e42152f5cf7b9"} err="failed to get container status \"467b61401d65c3d3cdc0f1a30801ad160061eeb907e9cb32955e42152f5cf7b9\": rpc error: code = NotFound desc = could not find container \"467b61401d65c3d3cdc0f1a30801ad160061eeb907e9cb32955e42152f5cf7b9\": container with ID starting with 467b61401d65c3d3cdc0f1a30801ad160061eeb907e9cb32955e42152f5cf7b9 not found: ID does not exist"
Jul 30 18:04:22.820949 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.820928    2577 scope.go:117] "RemoveContainer" containerID="51ebc9df2674bbc36b9af9b61605fbbaeab79d38100e92b6d4e91d77677a66d1"
Jul 30 18:04:22.822793 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.822774    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs"]
Jul 30 18:04:22.827349 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.827317    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c2052f19-9cf4fc6b6-dcdqs"]
Jul 30 18:04:22.829542 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.829528    2577 scope.go:117] "RemoveContainer" containerID="51ebc9df2674bbc36b9af9b61605fbbaeab79d38100e92b6d4e91d77677a66d1"
Jul 30 18:04:22.829827 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:04:22.829797    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51ebc9df2674bbc36b9af9b61605fbbaeab79d38100e92b6d4e91d77677a66d1\": container with ID starting with 51ebc9df2674bbc36b9af9b61605fbbaeab79d38100e92b6d4e91d77677a66d1 not found: ID does not exist" containerID="51ebc9df2674bbc36b9af9b61605fbbaeab79d38100e92b6d4e91d77677a66d1"
Jul 30 18:04:22.829936 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.829918    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51ebc9df2674bbc36b9af9b61605fbbaeab79d38100e92b6d4e91d77677a66d1"} err="failed to get container status \"51ebc9df2674bbc36b9af9b61605fbbaeab79d38100e92b6d4e91d77677a66d1\": rpc error: code = NotFound desc = could not find container \"51ebc9df2674bbc36b9af9b61605fbbaeab79d38100e92b6d4e91d77677a66d1\": container with ID starting with 51ebc9df2674bbc36b9af9b61605fbbaeab79d38100e92b6d4e91d77677a66d1 not found: ID does not exist"
Jul 30 18:04:22.838715 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.838299    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v"]
Jul 30 18:04:22.844449 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.844427    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c2052f19-58c8c75b88-b7b5v"]
Jul 30 18:04:22.854469 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.854446    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c"]
Jul 30 18:04:22.859425 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:22.859402    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c2052f19-7f7f487784-2q79c"]
Jul 30 18:04:23.076836 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:23.076784    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:04:23.309439 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:23.309403    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:04:23.530882 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:23.530846    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:04:23.853191 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:23.853105    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:04:23.948310 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:23.948269    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:04:24.111184 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:24.111102    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="079ca29d-267e-4cc1-80bc-e241193399b8" path="/var/lib/kubelet/pods/079ca29d-267e-4cc1-80bc-e241193399b8/volumes"
Jul 30 18:04:24.111476 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:24.111463    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69ed7543-df6d-4265-bfbc-655a62928418" path="/var/lib/kubelet/pods/69ed7543-df6d-4265-bfbc-655a62928418/volumes"
Jul 30 18:04:24.111765 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:24.111754    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a895919c-c08e-45ee-860b-49fb56806b32" path="/var/lib/kubelet/pods/a895919c-c08e-45ee-860b-49fb56806b32/volumes"
Jul 30 18:04:26.714265 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:26.714174    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:04:26.813691 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:26.813662    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:04:27.805922 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:27.805895    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n"]
Jul 30 18:04:27.806293 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:27.806280    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69ed7543-df6d-4265-bfbc-655a62928418" containerName="payload-pre-processing"
Jul 30 18:04:27.806380 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:27.806294    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="69ed7543-df6d-4265-bfbc-655a62928418" containerName="payload-pre-processing"
Jul 30 18:04:27.806380 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:27.806306    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="079ca29d-267e-4cc1-80bc-e241193399b8" containerName="maas-api"
Jul 30 18:04:27.806380 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:27.806312    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="079ca29d-267e-4cc1-80bc-e241193399b8" containerName="maas-api"
Jul 30 18:04:27.806380 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:27.806320    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a895919c-c08e-45ee-860b-49fb56806b32" containerName="payload-processing"
Jul 30 18:04:27.806380 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:27.806326    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a895919c-c08e-45ee-860b-49fb56806b32" containerName="payload-processing"
Jul 30 18:04:27.806526 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:27.806393    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="69ed7543-df6d-4265-bfbc-655a62928418" containerName="payload-pre-processing"
Jul 30 18:04:27.806526 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:27.806403    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="079ca29d-267e-4cc1-80bc-e241193399b8" containerName="maas-api"
Jul 30 18:04:27.806526 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:27.806408    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a895919c-c08e-45ee-860b-49fb56806b32" containerName="payload-processing"
Jul 30 18:04:27.810531 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:27.810510    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n"
Jul 30 18:04:27.812594 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:27.812575    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-9995abfe-serving-cert\""
Jul 30 18:04:27.820389 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:27.820363    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n"]
Jul 30 18:04:27.839357 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:27.839322    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:04:27.925795 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:27.925755    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flmkt\" (UniqueName: \"kubernetes.io/projected/2addd49d-46a4-4eff-a836-d39fe98e4801-kube-api-access-flmkt\") pod \"maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n\" (UID: \"2addd49d-46a4-4eff-a836-d39fe98e4801\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n"
Jul 30 18:04:27.929837 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:27.926298    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2addd49d-46a4-4eff-a836-d39fe98e4801-openshift-service-ca\") pod \"maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n\" (UID: \"2addd49d-46a4-4eff-a836-d39fe98e4801\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n"
Jul 30 18:04:27.932505 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:27.932456    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2addd49d-46a4-4eff-a836-d39fe98e4801-maas-api-tls\") pod \"maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n\" (UID: \"2addd49d-46a4-4eff-a836-d39fe98e4801\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n"
Jul 30 18:04:28.034083 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.034043    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2addd49d-46a4-4eff-a836-d39fe98e4801-openshift-service-ca\") pod \"maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n\" (UID: \"2addd49d-46a4-4eff-a836-d39fe98e4801\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n"
Jul 30 18:04:28.034282 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.034104    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2addd49d-46a4-4eff-a836-d39fe98e4801-maas-api-tls\") pod \"maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n\" (UID: \"2addd49d-46a4-4eff-a836-d39fe98e4801\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n"
Jul 30 18:04:28.034282 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.034150    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flmkt\" (UniqueName: \"kubernetes.io/projected/2addd49d-46a4-4eff-a836-d39fe98e4801-kube-api-access-flmkt\") pod \"maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n\" (UID: \"2addd49d-46a4-4eff-a836-d39fe98e4801\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n"
Jul 30 18:04:28.034282 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:04:28.034269    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-serving-cert: secret "maas-api-e2e-derive-9995abfe-serving-cert" not found
Jul 30 18:04:28.034446 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:04:28.034349    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2addd49d-46a4-4eff-a836-d39fe98e4801-maas-api-tls podName:2addd49d-46a4-4eff-a836-d39fe98e4801 nodeName:}" failed. No retries permitted until 2026-07-30 18:04:28.534330836 +0000 UTC m=+2242.987894805 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2addd49d-46a4-4eff-a836-d39fe98e4801-maas-api-tls") pod "maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n" (UID: "2addd49d-46a4-4eff-a836-d39fe98e4801") : secret "maas-api-e2e-derive-9995abfe-serving-cert" not found
Jul 30 18:04:28.034764 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.034741    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2addd49d-46a4-4eff-a836-d39fe98e4801-openshift-service-ca\") pod \"maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n\" (UID: \"2addd49d-46a4-4eff-a836-d39fe98e4801\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n"
Jul 30 18:04:28.043684 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.043657    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flmkt\" (UniqueName: \"kubernetes.io/projected/2addd49d-46a4-4eff-a836-d39fe98e4801-kube-api-access-flmkt\") pod \"maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n\" (UID: \"2addd49d-46a4-4eff-a836-d39fe98e4801\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n"
Jul 30 18:04:28.359250 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.359212    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:04:28.466106 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.466076    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:04:28.479035 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.479001    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp"]
Jul 30 18:04:28.482747 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.482728    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp"
Jul 30 18:04:28.486046 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.486028    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-9995abfe-dockercfg-zd4db\""
Jul 30 18:04:28.486147 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.486129    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-9995abfe\""
Jul 30 18:04:28.495432 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.495398    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp"]
Jul 30 18:04:28.499003 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.498964    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7"]
Jul 30 18:04:28.502439 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.502423    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7"
Jul 30 18:04:28.512232 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.512211    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7"]
Jul 30 18:04:28.538242 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.538215    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4xgk\" (UniqueName: \"kubernetes.io/projected/ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb-kube-api-access-p4xgk\") pod \"payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp\" (UID: \"ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp"
Jul 30 18:04:28.538367 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.538251    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2addd49d-46a4-4eff-a836-d39fe98e4801-maas-api-tls\") pod \"maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n\" (UID: \"2addd49d-46a4-4eff-a836-d39fe98e4801\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n"
Jul 30 18:04:28.538367 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.538276    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp\" (UID: \"ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp"
Jul 30 18:04:28.540683 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.540666    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2addd49d-46a4-4eff-a836-d39fe98e4801-maas-api-tls\") pod \"maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n\" (UID: \"2addd49d-46a4-4eff-a836-d39fe98e4801\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n"
Jul 30 18:04:28.639719 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.639628    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4xgk\" (UniqueName: \"kubernetes.io/projected/ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb-kube-api-access-p4xgk\") pod \"payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp\" (UID: \"ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp"
Jul 30 18:04:28.639719 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.639669    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fac33df-e031-4868-963d-fca8502a78e4-plugins-config-volume\") pod \"payload-processing-e2e-derive-9995abfe-786d448b58-zldg7\" (UID: \"6fac33df-e031-4868-963d-fca8502a78e4\") " pod="openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7"
Jul 30 18:04:28.639719 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.639688    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlgts\" (UniqueName: \"kubernetes.io/projected/6fac33df-e031-4868-963d-fca8502a78e4-kube-api-access-dlgts\") pod \"payload-processing-e2e-derive-9995abfe-786d448b58-zldg7\" (UID: \"6fac33df-e031-4868-963d-fca8502a78e4\") " pod="openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7"
Jul 30 18:04:28.639719 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.639720    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp\" (UID: \"ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp"
Jul 30 18:04:28.640388 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.640370    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp\" (UID: \"ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp"
Jul 30 18:04:28.648450 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.648425    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4xgk\" (UniqueName: \"kubernetes.io/projected/ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb-kube-api-access-p4xgk\") pod \"payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp\" (UID: \"ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp"
Jul 30 18:04:28.722421 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.722385    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n"
Jul 30 18:04:28.741099 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.741069    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlgts\" (UniqueName: \"kubernetes.io/projected/6fac33df-e031-4868-963d-fca8502a78e4-kube-api-access-dlgts\") pod \"payload-processing-e2e-derive-9995abfe-786d448b58-zldg7\" (UID: \"6fac33df-e031-4868-963d-fca8502a78e4\") " pod="openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7"
Jul 30 18:04:28.741245 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.741214    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fac33df-e031-4868-963d-fca8502a78e4-plugins-config-volume\") pod \"payload-processing-e2e-derive-9995abfe-786d448b58-zldg7\" (UID: \"6fac33df-e031-4868-963d-fca8502a78e4\") " pod="openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7"
Jul 30 18:04:28.741811 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.741792    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fac33df-e031-4868-963d-fca8502a78e4-plugins-config-volume\") pod \"payload-processing-e2e-derive-9995abfe-786d448b58-zldg7\" (UID: \"6fac33df-e031-4868-963d-fca8502a78e4\") " pod="openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7"
Jul 30 18:04:28.749510 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.749487    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlgts\" (UniqueName: \"kubernetes.io/projected/6fac33df-e031-4868-963d-fca8502a78e4-kube-api-access-dlgts\") pod \"payload-processing-e2e-derive-9995abfe-786d448b58-zldg7\" (UID: \"6fac33df-e031-4868-963d-fca8502a78e4\") " pod="openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7"
Jul 30 18:04:28.794660 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.794627    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp"
Jul 30 18:04:28.814637 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.814352    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7"
Jul 30 18:04:28.874365 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.873890    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n"]
Jul 30 18:04:28.879273 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:04:28.879238    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2addd49d_46a4_4eff_a836_d39fe98e4801.slice/crio-2706872a0fc15202312ca362cfbfb62229b5ca6805a52b333c182d96f648d4c4 WatchSource:0}: Error finding container 2706872a0fc15202312ca362cfbfb62229b5ca6805a52b333c182d96f648d4c4: Status 404 returned error can't find the container with id 2706872a0fc15202312ca362cfbfb62229b5ca6805a52b333c182d96f648d4c4
Jul 30 18:04:28.985504 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:28.985480    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7"]
Jul 30 18:04:28.987553 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:04:28.987525    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fac33df_e031_4868_963d_fca8502a78e4.slice/crio-b26718a050cebc7fd26b5b50d052e8cfbe75ed9717c096833e4757ba7d5ebda2 WatchSource:0}: Error finding container b26718a050cebc7fd26b5b50d052e8cfbe75ed9717c096833e4757ba7d5ebda2: Status 404 returned error can't find the container with id b26718a050cebc7fd26b5b50d052e8cfbe75ed9717c096833e4757ba7d5ebda2
Jul 30 18:04:29.164294 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:29.164260    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp"]
Jul 30 18:04:29.166112 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:04:29.166073    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce82bac6_4cbf_4f1e_b8b3_de7c298e83eb.slice/crio-656f77bc71285d5877bceec863d7cbaf240f9baf415d927cc91059f473cac283 WatchSource:0}: Error finding container 656f77bc71285d5877bceec863d7cbaf240f9baf415d927cc91059f473cac283: Status 404 returned error can't find the container with id 656f77bc71285d5877bceec863d7cbaf240f9baf415d927cc91059f473cac283
Jul 30 18:04:29.849902 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:29.849848    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp" event={"ID":"ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb","Type":"ContainerStarted","Data":"96f4d1925f6bc8ed6b6e1ffd7cfe271c5d638d95b36affe98377ded53ab31ba5"}
Jul 30 18:04:29.850329 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:29.849924    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp" event={"ID":"ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb","Type":"ContainerStarted","Data":"656f77bc71285d5877bceec863d7cbaf240f9baf415d927cc91059f473cac283"}
Jul 30 18:04:29.850329 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:29.850075    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp"
Jul 30 18:04:29.851911 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:29.851888    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7" event={"ID":"6fac33df-e031-4868-963d-fca8502a78e4","Type":"ContainerStarted","Data":"df935778570dfbe3b807df8d945b451085f1306ee1cc29fcccb9b37ec847d851"}
Jul 30 18:04:29.852052 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:29.851933    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7" event={"ID":"6fac33df-e031-4868-963d-fca8502a78e4","Type":"ContainerStarted","Data":"b26718a050cebc7fd26b5b50d052e8cfbe75ed9717c096833e4757ba7d5ebda2"}
Jul 30 18:04:29.852052 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:29.852010    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7"
Jul 30 18:04:29.853394 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:29.853376    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n" event={"ID":"2addd49d-46a4-4eff-a836-d39fe98e4801","Type":"ContainerStarted","Data":"e0d6ad64d612ea8004613d16ec608c739349629aa50cd059b462fbf5bee9bab5"}
Jul 30 18:04:29.853505 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:29.853407    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n" event={"ID":"2addd49d-46a4-4eff-a836-d39fe98e4801","Type":"ContainerStarted","Data":"2706872a0fc15202312ca362cfbfb62229b5ca6805a52b333c182d96f648d4c4"}
Jul 30 18:04:29.853548 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:29.853505    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n"
Jul 30 18:04:29.866339 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:29.866287    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp" podStartSLOduration=1.8662705370000001 podStartE2EDuration="1.866270537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:04:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:04:29.864216906 +0000 UTC m=+2244.317780896" watchObservedRunningTime="2026-07-30 18:04:29.866270537 +0000 UTC m=+2244.319834511"
Jul 30 18:04:29.880944 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:29.880909    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n" podStartSLOduration=2.462167162 podStartE2EDuration="2.88089378s" totalImagesPullingTime="418.726618ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:04:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:04:29.878622517 +0000 UTC m=+2244.332186506" watchObservedRunningTime="2026-07-30 18:04:29.88089378 +0000 UTC m=+2244.334457771"
Jul 30 18:04:29.893052 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:29.893014    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7" podStartSLOduration=1.893001136 podStartE2EDuration="1.893001136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:04:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:04:29.891867831 +0000 UTC m=+2244.345431821" watchObservedRunningTime="2026-07-30 18:04:29.893001136 +0000 UTC m=+2244.346565126"
Jul 30 18:04:35.864987 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:35.864956    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n"
Jul 30 18:04:40.861512 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:40.861476    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7"
Jul 30 18:04:40.862022 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:40.861566    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp"
Jul 30 18:04:58.387331 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.387296    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n"]
Jul 30 18:04:58.387796 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.387545    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n" podUID="2addd49d-46a4-4eff-a836-d39fe98e4801" containerName="maas-api" containerID="cri-o://e0d6ad64d612ea8004613d16ec608c739349629aa50cd059b462fbf5bee9bab5" gracePeriod=30
Jul 30 18:04:58.412956 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.412927    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:04:58.448125 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.448087    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7"]
Jul 30 18:04:58.448401 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.448375    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7" podUID="6fac33df-e031-4868-963d-fca8502a78e4" containerName="payload-processing" containerID="cri-o://df935778570dfbe3b807df8d945b451085f1306ee1cc29fcccb9b37ec847d851" gracePeriod=30
Jul 30 18:04:58.453324 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.452384    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp"]
Jul 30 18:04:58.453324 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.452602    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp" podUID="ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb" containerName="payload-pre-processing" containerID="cri-o://96f4d1925f6bc8ed6b6e1ffd7cfe271c5d638d95b36affe98377ded53ab31ba5" gracePeriod=30
Jul 30 18:04:58.489581 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.489550    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:04:58.571953 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.571919    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:04:58.640471 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.636060    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:04:58.743435 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.743404    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n"
Jul 30 18:04:58.770444 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.770408    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:04:58.772511 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.772483    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp"
Jul 30 18:04:58.789025 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.788996    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7"
Jul 30 18:04:58.835421 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.835389    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2addd49d-46a4-4eff-a836-d39fe98e4801-openshift-service-ca\") pod \"2addd49d-46a4-4eff-a836-d39fe98e4801\" (UID: \"2addd49d-46a4-4eff-a836-d39fe98e4801\") "
Jul 30 18:04:58.835421 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.835420    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fac33df-e031-4868-963d-fca8502a78e4-plugins-config-volume\") pod \"6fac33df-e031-4868-963d-fca8502a78e4\" (UID: \"6fac33df-e031-4868-963d-fca8502a78e4\") "
Jul 30 18:04:58.835624 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.835441    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flmkt\" (UniqueName: \"kubernetes.io/projected/2addd49d-46a4-4eff-a836-d39fe98e4801-kube-api-access-flmkt\") pod \"2addd49d-46a4-4eff-a836-d39fe98e4801\" (UID: \"2addd49d-46a4-4eff-a836-d39fe98e4801\") "
Jul 30 18:04:58.835624 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.835460    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlgts\" (UniqueName: \"kubernetes.io/projected/6fac33df-e031-4868-963d-fca8502a78e4-kube-api-access-dlgts\") pod \"6fac33df-e031-4868-963d-fca8502a78e4\" (UID: \"6fac33df-e031-4868-963d-fca8502a78e4\") "
Jul 30 18:04:58.835714 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.835660    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4xgk\" (UniqueName: \"kubernetes.io/projected/ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb-kube-api-access-p4xgk\") pod \"ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb\" (UID: \"ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb\") "
Jul 30 18:04:58.835801 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.835778    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fac33df-e031-4868-963d-fca8502a78e4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6fac33df-e031-4868-963d-fca8502a78e4" (UID: "6fac33df-e031-4868-963d-fca8502a78e4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:04:58.835890 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.835797    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2addd49d-46a4-4eff-a836-d39fe98e4801-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2addd49d-46a4-4eff-a836-d39fe98e4801" (UID: "2addd49d-46a4-4eff-a836-d39fe98e4801"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:04:58.835890 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.835786    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2addd49d-46a4-4eff-a836-d39fe98e4801-maas-api-tls\") pod \"2addd49d-46a4-4eff-a836-d39fe98e4801\" (UID: \"2addd49d-46a4-4eff-a836-d39fe98e4801\") "
Jul 30 18:04:58.835998 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.835895    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb-plugins-config-volume\") pod \"ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb\" (UID: \"ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb\") "
Jul 30 18:04:58.836365 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.836320    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb" (UID: "ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:04:58.836619 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.836522    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:04:58.836619 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.836543    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2addd49d-46a4-4eff-a836-d39fe98e4801-openshift-service-ca\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:04:58.836619 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.836558    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fac33df-e031-4868-963d-fca8502a78e4-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:04:58.837803 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.837776    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2addd49d-46a4-4eff-a836-d39fe98e4801-kube-api-access-flmkt" (OuterVolumeSpecName: "kube-api-access-flmkt") pod "2addd49d-46a4-4eff-a836-d39fe98e4801" (UID: "2addd49d-46a4-4eff-a836-d39fe98e4801"). InnerVolumeSpecName "kube-api-access-flmkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:04:58.837992 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.837953    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2addd49d-46a4-4eff-a836-d39fe98e4801-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2addd49d-46a4-4eff-a836-d39fe98e4801" (UID: "2addd49d-46a4-4eff-a836-d39fe98e4801"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 18:04:58.838256 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.838238    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fac33df-e031-4868-963d-fca8502a78e4-kube-api-access-dlgts" (OuterVolumeSpecName: "kube-api-access-dlgts") pod "6fac33df-e031-4868-963d-fca8502a78e4" (UID: "6fac33df-e031-4868-963d-fca8502a78e4"). InnerVolumeSpecName "kube-api-access-dlgts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:04:58.838346 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.838248    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb-kube-api-access-p4xgk" (OuterVolumeSpecName: "kube-api-access-p4xgk") pod "ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb" (UID: "ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb"). InnerVolumeSpecName "kube-api-access-p4xgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:04:58.937223 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.937141    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2addd49d-46a4-4eff-a836-d39fe98e4801-maas-api-tls\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:04:58.937223 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.937174    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flmkt\" (UniqueName: \"kubernetes.io/projected/2addd49d-46a4-4eff-a836-d39fe98e4801-kube-api-access-flmkt\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:04:58.937223 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.937187    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlgts\" (UniqueName: \"kubernetes.io/projected/6fac33df-e031-4868-963d-fca8502a78e4-kube-api-access-dlgts\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:04:58.937223 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:58.937197    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4xgk\" (UniqueName: \"kubernetes.io/projected/ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb-kube-api-access-p4xgk\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:04:59.012222 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.012188    2577 generic.go:356] "Generic (PLEG): container finished" podID="6fac33df-e031-4868-963d-fca8502a78e4" containerID="df935778570dfbe3b807df8d945b451085f1306ee1cc29fcccb9b37ec847d851" exitCode=0
Jul 30 18:04:59.012222 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.012227    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7"
Jul 30 18:04:59.012457 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.012264    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7" event={"ID":"6fac33df-e031-4868-963d-fca8502a78e4","Type":"ContainerDied","Data":"df935778570dfbe3b807df8d945b451085f1306ee1cc29fcccb9b37ec847d851"}
Jul 30 18:04:59.012457 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.012316    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7" event={"ID":"6fac33df-e031-4868-963d-fca8502a78e4","Type":"ContainerDied","Data":"b26718a050cebc7fd26b5b50d052e8cfbe75ed9717c096833e4757ba7d5ebda2"}
Jul 30 18:04:59.012457 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.012342    2577 scope.go:117] "RemoveContainer" containerID="df935778570dfbe3b807df8d945b451085f1306ee1cc29fcccb9b37ec847d851"
Jul 30 18:04:59.013740 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.013716    2577 generic.go:356] "Generic (PLEG): container finished" podID="2addd49d-46a4-4eff-a836-d39fe98e4801" containerID="e0d6ad64d612ea8004613d16ec608c739349629aa50cd059b462fbf5bee9bab5" exitCode=0
Jul 30 18:04:59.013888 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.013744    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n" event={"ID":"2addd49d-46a4-4eff-a836-d39fe98e4801","Type":"ContainerDied","Data":"e0d6ad64d612ea8004613d16ec608c739349629aa50cd059b462fbf5bee9bab5"}
Jul 30 18:04:59.013888 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.013776    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n"
Jul 30 18:04:59.013888 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.013809    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n" event={"ID":"2addd49d-46a4-4eff-a836-d39fe98e4801","Type":"ContainerDied","Data":"2706872a0fc15202312ca362cfbfb62229b5ca6805a52b333c182d96f648d4c4"}
Jul 30 18:04:59.014765 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.014744    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:04:59.015729 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.015697    2577 generic.go:356] "Generic (PLEG): container finished" podID="ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb" containerID="96f4d1925f6bc8ed6b6e1ffd7cfe271c5d638d95b36affe98377ded53ab31ba5" exitCode=0
Jul 30 18:04:59.015847 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.015745    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp" event={"ID":"ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb","Type":"ContainerDied","Data":"96f4d1925f6bc8ed6b6e1ffd7cfe271c5d638d95b36affe98377ded53ab31ba5"}
Jul 30 18:04:59.015847 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.015758    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp"
Jul 30 18:04:59.015847 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.015787    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp" event={"ID":"ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb","Type":"ContainerDied","Data":"656f77bc71285d5877bceec863d7cbaf240f9baf415d927cc91059f473cac283"}
Jul 30 18:04:59.023306 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.023290    2577 scope.go:117] "RemoveContainer" containerID="df935778570dfbe3b807df8d945b451085f1306ee1cc29fcccb9b37ec847d851"
Jul 30 18:04:59.023621 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:04:59.023591    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df935778570dfbe3b807df8d945b451085f1306ee1cc29fcccb9b37ec847d851\": container with ID starting with df935778570dfbe3b807df8d945b451085f1306ee1cc29fcccb9b37ec847d851 not found: ID does not exist" containerID="df935778570dfbe3b807df8d945b451085f1306ee1cc29fcccb9b37ec847d851"
Jul 30 18:04:59.023722 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.023632    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df935778570dfbe3b807df8d945b451085f1306ee1cc29fcccb9b37ec847d851"} err="failed to get container status \"df935778570dfbe3b807df8d945b451085f1306ee1cc29fcccb9b37ec847d851\": rpc error: code = NotFound desc = could not find container \"df935778570dfbe3b807df8d945b451085f1306ee1cc29fcccb9b37ec847d851\": container with ID starting with df935778570dfbe3b807df8d945b451085f1306ee1cc29fcccb9b37ec847d851 not found: ID does not exist"
Jul 30 18:04:59.023722 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.023658    2577 scope.go:117] "RemoveContainer" containerID="e0d6ad64d612ea8004613d16ec608c739349629aa50cd059b462fbf5bee9bab5"
Jul 30 18:04:59.036110 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.036085    2577 scope.go:117] "RemoveContainer" containerID="e0d6ad64d612ea8004613d16ec608c739349629aa50cd059b462fbf5bee9bab5"
Jul 30 18:04:59.036402 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:04:59.036381    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0d6ad64d612ea8004613d16ec608c739349629aa50cd059b462fbf5bee9bab5\": container with ID starting with e0d6ad64d612ea8004613d16ec608c739349629aa50cd059b462fbf5bee9bab5 not found: ID does not exist" containerID="e0d6ad64d612ea8004613d16ec608c739349629aa50cd059b462fbf5bee9bab5"
Jul 30 18:04:59.036447 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.036411    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0d6ad64d612ea8004613d16ec608c739349629aa50cd059b462fbf5bee9bab5"} err="failed to get container status \"e0d6ad64d612ea8004613d16ec608c739349629aa50cd059b462fbf5bee9bab5\": rpc error: code = NotFound desc = could not find container \"e0d6ad64d612ea8004613d16ec608c739349629aa50cd059b462fbf5bee9bab5\": container with ID starting with e0d6ad64d612ea8004613d16ec608c739349629aa50cd059b462fbf5bee9bab5 not found: ID does not exist"
Jul 30 18:04:59.036447 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.036428    2577 scope.go:117] "RemoveContainer" containerID="96f4d1925f6bc8ed6b6e1ffd7cfe271c5d638d95b36affe98377ded53ab31ba5"
Jul 30 18:04:59.044395 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.044365    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7"]
Jul 30 18:04:59.045438 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.045416    2577 scope.go:117] "RemoveContainer" containerID="96f4d1925f6bc8ed6b6e1ffd7cfe271c5d638d95b36affe98377ded53ab31ba5"
Jul 30 18:04:59.045711 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:04:59.045691    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96f4d1925f6bc8ed6b6e1ffd7cfe271c5d638d95b36affe98377ded53ab31ba5\": container with ID starting with 96f4d1925f6bc8ed6b6e1ffd7cfe271c5d638d95b36affe98377ded53ab31ba5 not found: ID does not exist" containerID="96f4d1925f6bc8ed6b6e1ffd7cfe271c5d638d95b36affe98377ded53ab31ba5"
Jul 30 18:04:59.045779 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.045721    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96f4d1925f6bc8ed6b6e1ffd7cfe271c5d638d95b36affe98377ded53ab31ba5"} err="failed to get container status \"96f4d1925f6bc8ed6b6e1ffd7cfe271c5d638d95b36affe98377ded53ab31ba5\": rpc error: code = NotFound desc = could not find container \"96f4d1925f6bc8ed6b6e1ffd7cfe271c5d638d95b36affe98377ded53ab31ba5\": container with ID starting with 96f4d1925f6bc8ed6b6e1ffd7cfe271c5d638d95b36affe98377ded53ab31ba5 not found: ID does not exist"
Jul 30 18:04:59.048346 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.048327    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9995abfe-786d448b58-zldg7"]
Jul 30 18:04:59.060712 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.060681    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp"]
Jul 30 18:04:59.064727 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.064704    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9995abfe-6d94ff6bd-d99mp"]
Jul 30 18:04:59.075029 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.074990    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n"]
Jul 30 18:04:59.079077 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.079052    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9995abfe-85cd9dd5b9-mqs5n"]
Jul 30 18:04:59.501170 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.501128    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:04:59.600541 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.600502    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:04:59.835177 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:04:59.835126    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:05:00.111490 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:05:00.111408    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2addd49d-46a4-4eff-a836-d39fe98e4801" path="/var/lib/kubelet/pods/2addd49d-46a4-4eff-a836-d39fe98e4801/volumes"
Jul 30 18:05:00.111993 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:05:00.111970    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fac33df-e031-4868-963d-fca8502a78e4" path="/var/lib/kubelet/pods/6fac33df-e031-4868-963d-fca8502a78e4/volumes"
Jul 30 18:05:00.112351 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:05:00.112333    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb" path="/var/lib/kubelet/pods/ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb/volumes"
Jul 30 18:05:21.692015 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:05:21.691953    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:05:28.268400 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:05:28.268367    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:05:28.374083 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:05:28.374054    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:05:28.493375 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:05:28.493340    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:05:28.588951 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:05:28.588810    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:05:28.994890 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:05:28.994775    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:05:55.035748 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:05:55.035706    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:06:01.655159 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:06:01.655075    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:06:01.774384 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:06:01.774353    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:06:01.911318 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:06:01.911239    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:06:17.834251 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:06:17.834219    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:06:23.903670 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:06:23.903635    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:06:23.970925 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:06:23.970898    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:06:24.075339 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:06:24.075305    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:06:24.172792 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:06:24.172713    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:06:34.667011 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:06:34.666982    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:06:45.736183 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:06:45.736146    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:06:52.646905 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:06:52.646869    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:06:52.753511 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:06:52.753483    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:06:52.848085 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:06:52.848052    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:06:52.948047 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:06:52.947963    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:06:53.687935 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:06:53.687876    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:06:54.376718 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:06:54.376666    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:06:55.364249 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:06:55.364215    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:01.566077 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:01.566004    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:01.666585 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:01.666531    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:02.162305 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:02.162238    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:08.644952 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:08.644916    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:08.747381 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:08.747322    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:08.922435 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:08.922352    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:09.339734 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:09.339701    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:09.901014 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:09.900976    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:12.141660 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:12.141604    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:12.239249 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:12.239220    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:13.236925 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.236895    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j"]
Jul 30 18:07:13.237302 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.237290    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb" containerName="payload-pre-processing"
Jul 30 18:07:13.237346 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.237304    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb" containerName="payload-pre-processing"
Jul 30 18:07:13.237346 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.237312    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fac33df-e031-4868-963d-fca8502a78e4" containerName="payload-processing"
Jul 30 18:07:13.237346 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.237317    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fac33df-e031-4868-963d-fca8502a78e4" containerName="payload-processing"
Jul 30 18:07:13.237346 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.237336    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2addd49d-46a4-4eff-a836-d39fe98e4801" containerName="maas-api"
Jul 30 18:07:13.237346 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.237341    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2addd49d-46a4-4eff-a836-d39fe98e4801" containerName="maas-api"
Jul 30 18:07:13.237501 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.237404    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2addd49d-46a4-4eff-a836-d39fe98e4801" containerName="maas-api"
Jul 30 18:07:13.237501 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.237413    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fac33df-e031-4868-963d-fca8502a78e4" containerName="payload-processing"
Jul 30 18:07:13.237501 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.237418    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce82bac6-4cbf-4f1e-b8b3-de7c298e83eb" containerName="payload-pre-processing"
Jul 30 18:07:13.240684 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.240602    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j"
Jul 30 18:07:13.242809 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.242781    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e004b9c2-serving-cert\""
Jul 30 18:07:13.250664 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.250636    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j"]
Jul 30 18:07:13.281147 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.281115    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:13.400761 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.400722    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn4xr\" (UniqueName: \"kubernetes.io/projected/f251180b-33f4-4ea9-961c-80327d038aba-kube-api-access-bn4xr\") pod \"maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j\" (UID: \"f251180b-33f4-4ea9-961c-80327d038aba\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j"
Jul 30 18:07:13.400946 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.400769    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f251180b-33f4-4ea9-961c-80327d038aba-openshift-service-ca\") pod \"maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j\" (UID: \"f251180b-33f4-4ea9-961c-80327d038aba\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j"
Jul 30 18:07:13.400946 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.400806    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f251180b-33f4-4ea9-961c-80327d038aba-maas-api-tls\") pod \"maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j\" (UID: \"f251180b-33f4-4ea9-961c-80327d038aba\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j"
Jul 30 18:07:13.502096 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.502067    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn4xr\" (UniqueName: \"kubernetes.io/projected/f251180b-33f4-4ea9-961c-80327d038aba-kube-api-access-bn4xr\") pod \"maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j\" (UID: \"f251180b-33f4-4ea9-961c-80327d038aba\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j"
Jul 30 18:07:13.502281 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.502109    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f251180b-33f4-4ea9-961c-80327d038aba-openshift-service-ca\") pod \"maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j\" (UID: \"f251180b-33f4-4ea9-961c-80327d038aba\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j"
Jul 30 18:07:13.502281 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.502134    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f251180b-33f4-4ea9-961c-80327d038aba-maas-api-tls\") pod \"maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j\" (UID: \"f251180b-33f4-4ea9-961c-80327d038aba\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j"
Jul 30 18:07:13.502864 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.502844    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f251180b-33f4-4ea9-961c-80327d038aba-openshift-service-ca\") pod \"maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j\" (UID: \"f251180b-33f4-4ea9-961c-80327d038aba\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j"
Jul 30 18:07:13.504908 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.504886    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f251180b-33f4-4ea9-961c-80327d038aba-maas-api-tls\") pod \"maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j\" (UID: \"f251180b-33f4-4ea9-961c-80327d038aba\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j"
Jul 30 18:07:13.510699 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.510677    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn4xr\" (UniqueName: \"kubernetes.io/projected/f251180b-33f4-4ea9-961c-80327d038aba-kube-api-access-bn4xr\") pod \"maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j\" (UID: \"f251180b-33f4-4ea9-961c-80327d038aba\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j"
Jul 30 18:07:13.552556 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.552509    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j"
Jul 30 18:07:13.553506 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.553477    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:13.811222 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.811132    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:13.911517 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.911457    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j"]
Jul 30 18:07:13.913798 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.913751    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:13.914386 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:07:13.914361    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf251180b_33f4_4ea9_961c_80327d038aba.slice/crio-4afe28653a3ca4a61ce7fea8cce57f74a276b9e3b91bd07f25570dbc09dccc5f WatchSource:0}: Error finding container 4afe28653a3ca4a61ce7fea8cce57f74a276b9e3b91bd07f25570dbc09dccc5f: Status 404 returned error can't find the container with id 4afe28653a3ca4a61ce7fea8cce57f74a276b9e3b91bd07f25570dbc09dccc5f
Jul 30 18:07:13.930332 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.930308    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8"]
Jul 30 18:07:13.934919 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.934898    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8"
Jul 30 18:07:13.937491 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.937470    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-e004b9c2-dockercfg-9ptj8\""
Jul 30 18:07:13.937602 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.937544    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-e004b9c2\""
Jul 30 18:07:13.946123 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:13.946100    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8"]
Jul 30 18:07:14.006164 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:14.006133    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6072dd3-33b7-4940-9de1-cff81f909b14-plugins-config-volume\") pod \"payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8\" (UID: \"e6072dd3-33b7-4940-9de1-cff81f909b14\") " pod="openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8"
Jul 30 18:07:14.006334 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:14.006193    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kf6d\" (UniqueName: \"kubernetes.io/projected/e6072dd3-33b7-4940-9de1-cff81f909b14-kube-api-access-5kf6d\") pod \"payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8\" (UID: \"e6072dd3-33b7-4940-9de1-cff81f909b14\") " pod="openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8"
Jul 30 18:07:14.107421 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:14.107337    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kf6d\" (UniqueName: \"kubernetes.io/projected/e6072dd3-33b7-4940-9de1-cff81f909b14-kube-api-access-5kf6d\") pod \"payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8\" (UID: \"e6072dd3-33b7-4940-9de1-cff81f909b14\") " pod="openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8"
Jul 30 18:07:14.107577 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:14.107530    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6072dd3-33b7-4940-9de1-cff81f909b14-plugins-config-volume\") pod \"payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8\" (UID: \"e6072dd3-33b7-4940-9de1-cff81f909b14\") " pod="openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8"
Jul 30 18:07:14.108387 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:14.108364    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6072dd3-33b7-4940-9de1-cff81f909b14-plugins-config-volume\") pod \"payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8\" (UID: \"e6072dd3-33b7-4940-9de1-cff81f909b14\") " pod="openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8"
Jul 30 18:07:14.115490 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:14.115464    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kf6d\" (UniqueName: \"kubernetes.io/projected/e6072dd3-33b7-4940-9de1-cff81f909b14-kube-api-access-5kf6d\") pod \"payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8\" (UID: \"e6072dd3-33b7-4940-9de1-cff81f909b14\") " pod="openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8"
Jul 30 18:07:14.248019 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:14.247981    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8"
Jul 30 18:07:14.398102 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:14.398075    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8"]
Jul 30 18:07:14.400266 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:07:14.400238    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6072dd3_33b7_4940_9de1_cff81f909b14.slice/crio-f7703248adbeb56f25986c1c686690e260e45a64f6c44c81aefa3be8f65f11dc WatchSource:0}: Error finding container f7703248adbeb56f25986c1c686690e260e45a64f6c44c81aefa3be8f65f11dc: Status 404 returned error can't find the container with id f7703248adbeb56f25986c1c686690e260e45a64f6c44c81aefa3be8f65f11dc
Jul 30 18:07:14.724711 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:14.724621    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j" event={"ID":"f251180b-33f4-4ea9-961c-80327d038aba","Type":"ContainerStarted","Data":"9db80df381c5713d134fb34f658fc2643df77c9d1292961b1ba6b9a12ababb04"}
Jul 30 18:07:14.724711 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:14.724679    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j" event={"ID":"f251180b-33f4-4ea9-961c-80327d038aba","Type":"ContainerStarted","Data":"4afe28653a3ca4a61ce7fea8cce57f74a276b9e3b91bd07f25570dbc09dccc5f"}
Jul 30 18:07:14.724975 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:14.724727    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j"
Jul 30 18:07:14.726433 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:14.726411    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8" event={"ID":"e6072dd3-33b7-4940-9de1-cff81f909b14","Type":"ContainerStarted","Data":"8c4918e6bbd07426815b92c15d75e0cad4be75b53c53beee7170b452ad109596"}
Jul 30 18:07:14.726529 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:14.726449    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8" event={"ID":"e6072dd3-33b7-4940-9de1-cff81f909b14","Type":"ContainerStarted","Data":"f7703248adbeb56f25986c1c686690e260e45a64f6c44c81aefa3be8f65f11dc"}
Jul 30 18:07:14.726529 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:14.726470    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8"
Jul 30 18:07:14.742526 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:14.742483    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j" podStartSLOduration=1.361701553 podStartE2EDuration="1.742471291s" totalImagesPullingTime="380.769738ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:07:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:07:14.740626703 +0000 UTC m=+2409.194190690" watchObservedRunningTime="2026-07-30 18:07:14.742471291 +0000 UTC m=+2409.196035282"
Jul 30 18:07:14.764589 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:14.764531    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8" podStartSLOduration=1.76451342 podStartE2EDuration="1.76451342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:07:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:07:14.763224198 +0000 UTC m=+2409.216788190" watchObservedRunningTime="2026-07-30 18:07:14.76451342 +0000 UTC m=+2409.218077412"
Jul 30 18:07:20.737325 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:20.737295    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j"
Jul 30 18:07:25.734435 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:25.734403    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8"
Jul 30 18:07:37.163491 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:37.163457    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:37.854356 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:37.854316    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:37.951221 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:37.951189    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:38.287945 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:38.287918    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:44.366806 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:44.366750    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:44.468408 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:44.468370    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:55.311644 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.311609    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j"]
Jul 30 18:07:55.313266 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.313197    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j" podUID="f251180b-33f4-4ea9-961c-80327d038aba" containerName="maas-api" containerID="cri-o://9db80df381c5713d134fb34f658fc2643df77c9d1292961b1ba6b9a12ababb04" gracePeriod=30
Jul 30 18:07:55.343974 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.343945    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:55.372560 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.372508    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8"]
Jul 30 18:07:55.372885 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.372777    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8" podUID="e6072dd3-33b7-4940-9de1-cff81f909b14" containerName="payload-processing" containerID="cri-o://8c4918e6bbd07426815b92c15d75e0cad4be75b53c53beee7170b452ad109596" gracePeriod=30
Jul 30 18:07:55.447624 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.447591    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:55.546935 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.546905    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:55.622284 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.622259    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j"
Jul 30 18:07:55.645798 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.645770    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:55.651836 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.651799    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8"
Jul 30 18:07:55.730135 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.730106    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f251180b-33f4-4ea9-961c-80327d038aba-maas-api-tls\") pod \"f251180b-33f4-4ea9-961c-80327d038aba\" (UID: \"f251180b-33f4-4ea9-961c-80327d038aba\") "
Jul 30 18:07:55.730305 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.730154    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bn4xr\" (UniqueName: \"kubernetes.io/projected/f251180b-33f4-4ea9-961c-80327d038aba-kube-api-access-bn4xr\") pod \"f251180b-33f4-4ea9-961c-80327d038aba\" (UID: \"f251180b-33f4-4ea9-961c-80327d038aba\") "
Jul 30 18:07:55.730305 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.730221    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f251180b-33f4-4ea9-961c-80327d038aba-openshift-service-ca\") pod \"f251180b-33f4-4ea9-961c-80327d038aba\" (UID: \"f251180b-33f4-4ea9-961c-80327d038aba\") "
Jul 30 18:07:55.730626 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.730604    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f251180b-33f4-4ea9-961c-80327d038aba-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f251180b-33f4-4ea9-961c-80327d038aba" (UID: "f251180b-33f4-4ea9-961c-80327d038aba"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:07:55.732431 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.732412    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f251180b-33f4-4ea9-961c-80327d038aba-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f251180b-33f4-4ea9-961c-80327d038aba" (UID: "f251180b-33f4-4ea9-961c-80327d038aba"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 18:07:55.732486 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.732428    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f251180b-33f4-4ea9-961c-80327d038aba-kube-api-access-bn4xr" (OuterVolumeSpecName: "kube-api-access-bn4xr") pod "f251180b-33f4-4ea9-961c-80327d038aba" (UID: "f251180b-33f4-4ea9-961c-80327d038aba"). InnerVolumeSpecName "kube-api-access-bn4xr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:07:55.831285 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.831183    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6072dd3-33b7-4940-9de1-cff81f909b14-plugins-config-volume\") pod \"e6072dd3-33b7-4940-9de1-cff81f909b14\" (UID: \"e6072dd3-33b7-4940-9de1-cff81f909b14\") "
Jul 30 18:07:55.831285 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.831275    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kf6d\" (UniqueName: \"kubernetes.io/projected/e6072dd3-33b7-4940-9de1-cff81f909b14-kube-api-access-5kf6d\") pod \"e6072dd3-33b7-4940-9de1-cff81f909b14\" (UID: \"e6072dd3-33b7-4940-9de1-cff81f909b14\") "
Jul 30 18:07:55.831555 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.831524    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6072dd3-33b7-4940-9de1-cff81f909b14-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e6072dd3-33b7-4940-9de1-cff81f909b14" (UID: "e6072dd3-33b7-4940-9de1-cff81f909b14"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:07:55.831555 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.831549    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f251180b-33f4-4ea9-961c-80327d038aba-maas-api-tls\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:07:55.831653 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.831570    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bn4xr\" (UniqueName: \"kubernetes.io/projected/f251180b-33f4-4ea9-961c-80327d038aba-kube-api-access-bn4xr\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:07:55.831653 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.831586    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f251180b-33f4-4ea9-961c-80327d038aba-openshift-service-ca\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:07:55.833457 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.833432    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6072dd3-33b7-4940-9de1-cff81f909b14-kube-api-access-5kf6d" (OuterVolumeSpecName: "kube-api-access-5kf6d") pod "e6072dd3-33b7-4940-9de1-cff81f909b14" (UID: "e6072dd3-33b7-4940-9de1-cff81f909b14"). InnerVolumeSpecName "kube-api-access-5kf6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:07:55.932303 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.932257    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6072dd3-33b7-4940-9de1-cff81f909b14-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:07:55.932303 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.932296    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kf6d\" (UniqueName: \"kubernetes.io/projected/e6072dd3-33b7-4940-9de1-cff81f909b14-kube-api-access-5kf6d\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:07:55.953252 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.953218    2577 generic.go:356] "Generic (PLEG): container finished" podID="f251180b-33f4-4ea9-961c-80327d038aba" containerID="9db80df381c5713d134fb34f658fc2643df77c9d1292961b1ba6b9a12ababb04" exitCode=0
Jul 30 18:07:55.953419 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.953265    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j"
Jul 30 18:07:55.953419 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.953310    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j" event={"ID":"f251180b-33f4-4ea9-961c-80327d038aba","Type":"ContainerDied","Data":"9db80df381c5713d134fb34f658fc2643df77c9d1292961b1ba6b9a12ababb04"}
Jul 30 18:07:55.953419 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.953383    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j" event={"ID":"f251180b-33f4-4ea9-961c-80327d038aba","Type":"ContainerDied","Data":"4afe28653a3ca4a61ce7fea8cce57f74a276b9e3b91bd07f25570dbc09dccc5f"}
Jul 30 18:07:55.953419 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.953418    2577 scope.go:117] "RemoveContainer" containerID="9db80df381c5713d134fb34f658fc2643df77c9d1292961b1ba6b9a12ababb04"
Jul 30 18:07:55.954670 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.954649    2577 generic.go:356] "Generic (PLEG): container finished" podID="e6072dd3-33b7-4940-9de1-cff81f909b14" containerID="8c4918e6bbd07426815b92c15d75e0cad4be75b53c53beee7170b452ad109596" exitCode=0
Jul 30 18:07:55.954742 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.954692    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8"
Jul 30 18:07:55.954742 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.954725    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8" event={"ID":"e6072dd3-33b7-4940-9de1-cff81f909b14","Type":"ContainerDied","Data":"8c4918e6bbd07426815b92c15d75e0cad4be75b53c53beee7170b452ad109596"}
Jul 30 18:07:55.954840 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.954755    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8" event={"ID":"e6072dd3-33b7-4940-9de1-cff81f909b14","Type":"ContainerDied","Data":"f7703248adbeb56f25986c1c686690e260e45a64f6c44c81aefa3be8f65f11dc"}
Jul 30 18:07:55.964299 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.964277    2577 scope.go:117] "RemoveContainer" containerID="9db80df381c5713d134fb34f658fc2643df77c9d1292961b1ba6b9a12ababb04"
Jul 30 18:07:55.964635 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:07:55.964608    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9db80df381c5713d134fb34f658fc2643df77c9d1292961b1ba6b9a12ababb04\": container with ID starting with 9db80df381c5713d134fb34f658fc2643df77c9d1292961b1ba6b9a12ababb04 not found: ID does not exist" containerID="9db80df381c5713d134fb34f658fc2643df77c9d1292961b1ba6b9a12ababb04"
Jul 30 18:07:55.964749 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.964646    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9db80df381c5713d134fb34f658fc2643df77c9d1292961b1ba6b9a12ababb04"} err="failed to get container status \"9db80df381c5713d134fb34f658fc2643df77c9d1292961b1ba6b9a12ababb04\": rpc error: code = NotFound desc = could not find container \"9db80df381c5713d134fb34f658fc2643df77c9d1292961b1ba6b9a12ababb04\": container with ID starting with 9db80df381c5713d134fb34f658fc2643df77c9d1292961b1ba6b9a12ababb04 not found: ID does not exist"
Jul 30 18:07:55.964749 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.964670    2577 scope.go:117] "RemoveContainer" containerID="8c4918e6bbd07426815b92c15d75e0cad4be75b53c53beee7170b452ad109596"
Jul 30 18:07:55.973023 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.973004    2577 scope.go:117] "RemoveContainer" containerID="8c4918e6bbd07426815b92c15d75e0cad4be75b53c53beee7170b452ad109596"
Jul 30 18:07:55.973279 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:07:55.973263    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c4918e6bbd07426815b92c15d75e0cad4be75b53c53beee7170b452ad109596\": container with ID starting with 8c4918e6bbd07426815b92c15d75e0cad4be75b53c53beee7170b452ad109596 not found: ID does not exist" containerID="8c4918e6bbd07426815b92c15d75e0cad4be75b53c53beee7170b452ad109596"
Jul 30 18:07:55.973347 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.973286    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c4918e6bbd07426815b92c15d75e0cad4be75b53c53beee7170b452ad109596"} err="failed to get container status \"8c4918e6bbd07426815b92c15d75e0cad4be75b53c53beee7170b452ad109596\": rpc error: code = NotFound desc = could not find container \"8c4918e6bbd07426815b92c15d75e0cad4be75b53c53beee7170b452ad109596\": container with ID starting with 8c4918e6bbd07426815b92c15d75e0cad4be75b53c53beee7170b452ad109596 not found: ID does not exist"
Jul 30 18:07:55.979219 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.979199    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:07:55.979893 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.979876    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j"]
Jul 30 18:07:55.984683 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.984663    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e004b9c2-75856f68dd-glf5j"]
Jul 30 18:07:55.995628 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.995607    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8"]
Jul 30 18:07:55.999680 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:55.999659    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e004b9c2-7889c487bb-2n6x8"]
Jul 30 18:07:56.112201 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:56.112124    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6072dd3-33b7-4940-9de1-cff81f909b14" path="/var/lib/kubelet/pods/e6072dd3-33b7-4940-9de1-cff81f909b14/volumes"
Jul 30 18:07:56.112493 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:07:56.112481    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f251180b-33f4-4ea9-961c-80327d038aba" path="/var/lib/kubelet/pods/f251180b-33f4-4ea9-961c-80327d038aba/volumes"
Jul 30 18:08:02.419005 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:02.418962    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:08:02.521541 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:02.521509    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:08:02.670975 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:02.670880    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:08:02.891783 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:02.891751    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:08:03.022982 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:03.022952    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:08:03.125487 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:03.125454    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:08:03.251257 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:03.251221    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:08:09.347298 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:09.347262    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:08:09.412726 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:09.412696    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:08:09.562104 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:09.562067    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:08:09.858308 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:09.858277    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:08:09.923530 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:09.923499    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:08:15.966122 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:15.966083    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:08:16.043973 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:16.043945    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:08:44.446331 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:44.446293    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l"]
Jul 30 18:08:44.446900 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:44.446515    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l" podUID="60d5eb12-9d1d-4233-a4d9-f2cc305882c0" containerName="manager" containerID="cri-o://7c846d4de370f316ccc14b7607f8ab00b577a0bb4f3b43216d7e834a55d6667f" gracePeriod=10
Jul 30 18:08:45.695980 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:45.695957    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l"
Jul 30 18:08:45.879332 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:45.879295    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcs75\" (UniqueName: \"kubernetes.io/projected/60d5eb12-9d1d-4233-a4d9-f2cc305882c0-kube-api-access-fcs75\") pod \"60d5eb12-9d1d-4233-a4d9-f2cc305882c0\" (UID: \"60d5eb12-9d1d-4233-a4d9-f2cc305882c0\") "
Jul 30 18:08:45.879512 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:45.879354    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/60d5eb12-9d1d-4233-a4d9-f2cc305882c0-extensions-socket-volume\") pod \"60d5eb12-9d1d-4233-a4d9-f2cc305882c0\" (UID: \"60d5eb12-9d1d-4233-a4d9-f2cc305882c0\") "
Jul 30 18:08:45.879747 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:45.879722    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60d5eb12-9d1d-4233-a4d9-f2cc305882c0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "60d5eb12-9d1d-4233-a4d9-f2cc305882c0" (UID: "60d5eb12-9d1d-4233-a4d9-f2cc305882c0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 18:08:45.881609 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:45.881586    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60d5eb12-9d1d-4233-a4d9-f2cc305882c0-kube-api-access-fcs75" (OuterVolumeSpecName: "kube-api-access-fcs75") pod "60d5eb12-9d1d-4233-a4d9-f2cc305882c0" (UID: "60d5eb12-9d1d-4233-a4d9-f2cc305882c0"). InnerVolumeSpecName "kube-api-access-fcs75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:08:45.980184 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:45.980149    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcs75\" (UniqueName: \"kubernetes.io/projected/60d5eb12-9d1d-4233-a4d9-f2cc305882c0-kube-api-access-fcs75\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:08:45.980184 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:45.980180    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/60d5eb12-9d1d-4233-a4d9-f2cc305882c0-extensions-socket-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:08:46.217421 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:46.217326    2577 generic.go:356] "Generic (PLEG): container finished" podID="60d5eb12-9d1d-4233-a4d9-f2cc305882c0" containerID="7c846d4de370f316ccc14b7607f8ab00b577a0bb4f3b43216d7e834a55d6667f" exitCode=0
Jul 30 18:08:46.217421 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:46.217372    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l"
Jul 30 18:08:46.217421 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:46.217412    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l" event={"ID":"60d5eb12-9d1d-4233-a4d9-f2cc305882c0","Type":"ContainerDied","Data":"7c846d4de370f316ccc14b7607f8ab00b577a0bb4f3b43216d7e834a55d6667f"}
Jul 30 18:08:46.217695 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:46.217470    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l" event={"ID":"60d5eb12-9d1d-4233-a4d9-f2cc305882c0","Type":"ContainerDied","Data":"e069563ea00856ac5393390382c45e3860b332e47aa10e70a6592823b371ab97"}
Jul 30 18:08:46.217695 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:46.217494    2577 scope.go:117] "RemoveContainer" containerID="7c846d4de370f316ccc14b7607f8ab00b577a0bb4f3b43216d7e834a55d6667f"
Jul 30 18:08:46.227672 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:46.227653    2577 scope.go:117] "RemoveContainer" containerID="7c846d4de370f316ccc14b7607f8ab00b577a0bb4f3b43216d7e834a55d6667f"
Jul 30 18:08:46.227938 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:08:46.227914    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c846d4de370f316ccc14b7607f8ab00b577a0bb4f3b43216d7e834a55d6667f\": container with ID starting with 7c846d4de370f316ccc14b7607f8ab00b577a0bb4f3b43216d7e834a55d6667f not found: ID does not exist" containerID="7c846d4de370f316ccc14b7607f8ab00b577a0bb4f3b43216d7e834a55d6667f"
Jul 30 18:08:46.228024 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:46.227949    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c846d4de370f316ccc14b7607f8ab00b577a0bb4f3b43216d7e834a55d6667f"} err="failed to get container status \"7c846d4de370f316ccc14b7607f8ab00b577a0bb4f3b43216d7e834a55d6667f\": rpc error: code = NotFound desc = could not find container \"7c846d4de370f316ccc14b7607f8ab00b577a0bb4f3b43216d7e834a55d6667f\": container with ID starting with 7c846d4de370f316ccc14b7607f8ab00b577a0bb4f3b43216d7e834a55d6667f not found: ID does not exist"
Jul 30 18:08:46.235551 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:46.235527    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l"]
Jul 30 18:08:46.239458 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:46.239432    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rp4l"]
Jul 30 18:08:48.110382 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:08:48.110349    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60d5eb12-9d1d-4233-a4d9-f2cc305882c0" path="/var/lib/kubelet/pods/60d5eb12-9d1d-4233-a4d9-f2cc305882c0/volumes"
Jul 30 18:09:25.089493 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:09:25.089457    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:09:27.360364 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:09:27.360326    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:09:27.426379 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:09:27.426341    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:10:00.146974 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:00.146932    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xmfjp"]
Jul 30 18:10:00.147439 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:00.147355    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60d5eb12-9d1d-4233-a4d9-f2cc305882c0" containerName="manager"
Jul 30 18:10:00.147439 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:00.147366    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="60d5eb12-9d1d-4233-a4d9-f2cc305882c0" containerName="manager"
Jul 30 18:10:00.147439 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:00.147386    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f251180b-33f4-4ea9-961c-80327d038aba" containerName="maas-api"
Jul 30 18:10:00.147439 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:00.147392    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f251180b-33f4-4ea9-961c-80327d038aba" containerName="maas-api"
Jul 30 18:10:00.147439 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:00.147401    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6072dd3-33b7-4940-9de1-cff81f909b14" containerName="payload-processing"
Jul 30 18:10:00.147439 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:00.147410    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6072dd3-33b7-4940-9de1-cff81f909b14" containerName="payload-processing"
Jul 30 18:10:00.147631 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:00.147492    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6072dd3-33b7-4940-9de1-cff81f909b14" containerName="payload-processing"
Jul 30 18:10:00.147631 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:00.147502    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="60d5eb12-9d1d-4233-a4d9-f2cc305882c0" containerName="manager"
Jul 30 18:10:00.147631 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:00.147510    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f251180b-33f4-4ea9-961c-80327d038aba" containerName="maas-api"
Jul 30 18:10:00.150683 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:00.150666    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xmfjp"
Jul 30 18:10:00.153678 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:00.153660    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-lw975\""
Jul 30 18:10:00.164334 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:00.164295    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xmfjp"]
Jul 30 18:10:00.268283 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:00.268248    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f07c466d-53a4-4416-94a3-4bef276fb42f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xmfjp\" (UID: \"f07c466d-53a4-4416-94a3-4bef276fb42f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xmfjp"
Jul 30 18:10:00.268475 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:00.268328    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6vwk\" (UniqueName: \"kubernetes.io/projected/f07c466d-53a4-4416-94a3-4bef276fb42f-kube-api-access-p6vwk\") pod \"kuadrant-operator-controller-manager-86878c769f-xmfjp\" (UID: \"f07c466d-53a4-4416-94a3-4bef276fb42f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xmfjp"
Jul 30 18:10:00.369153 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:00.369112    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f07c466d-53a4-4416-94a3-4bef276fb42f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xmfjp\" (UID: \"f07c466d-53a4-4416-94a3-4bef276fb42f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xmfjp"
Jul 30 18:10:00.369340 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:00.369190    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6vwk\" (UniqueName: \"kubernetes.io/projected/f07c466d-53a4-4416-94a3-4bef276fb42f-kube-api-access-p6vwk\") pod \"kuadrant-operator-controller-manager-86878c769f-xmfjp\" (UID: \"f07c466d-53a4-4416-94a3-4bef276fb42f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xmfjp"
Jul 30 18:10:00.369498 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:00.369478    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f07c466d-53a4-4416-94a3-4bef276fb42f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xmfjp\" (UID: \"f07c466d-53a4-4416-94a3-4bef276fb42f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xmfjp"
Jul 30 18:10:00.378844 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:00.378792    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6vwk\" (UniqueName: \"kubernetes.io/projected/f07c466d-53a4-4416-94a3-4bef276fb42f-kube-api-access-p6vwk\") pod \"kuadrant-operator-controller-manager-86878c769f-xmfjp\" (UID: \"f07c466d-53a4-4416-94a3-4bef276fb42f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xmfjp"
Jul 30 18:10:00.468904 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:00.468799    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xmfjp"
Jul 30 18:10:00.610038 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:00.609886    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xmfjp"]
Jul 30 18:10:00.612763 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:10:00.612733    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf07c466d_53a4_4416_94a3_4bef276fb42f.slice/crio-a537e231f101792ca793ed2b8080c062eef0f00e38e621c3e74eee20b610ecfd WatchSource:0}: Error finding container a537e231f101792ca793ed2b8080c062eef0f00e38e621c3e74eee20b610ecfd: Status 404 returned error can't find the container with id a537e231f101792ca793ed2b8080c062eef0f00e38e621c3e74eee20b610ecfd
Jul 30 18:10:01.596494 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:01.596462    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xmfjp" event={"ID":"f07c466d-53a4-4416-94a3-4bef276fb42f","Type":"ContainerStarted","Data":"5133272828e5e90a36094e2095ac06fa87d1dd928c0ba84434c951c618a3e2ce"}
Jul 30 18:10:01.596922 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:01.596511    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xmfjp" event={"ID":"f07c466d-53a4-4416-94a3-4bef276fb42f","Type":"ContainerStarted","Data":"a537e231f101792ca793ed2b8080c062eef0f00e38e621c3e74eee20b610ecfd"}
Jul 30 18:10:01.596922 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:01.596597    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xmfjp"
Jul 30 18:10:01.616702 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:01.616647    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xmfjp" podStartSLOduration=1.616630079 podStartE2EDuration="1.616630079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:10:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:10:01.614684024 +0000 UTC m=+2576.068248016" watchObservedRunningTime="2026-07-30 18:10:01.616630079 +0000 UTC m=+2576.070194069"
Jul 30 18:10:12.604496 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:12.604466    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xmfjp"
Jul 30 18:10:19.187325 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:19.187290    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:10:19.516047 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:19.516008    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:10:19.720967 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:19.720937    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:10:22.169149 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:22.169112    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:10:22.261451 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:22.261417    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:10:23.266612 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.266575    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9"]
Jul 30 18:10:23.270874 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.270851    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9"
Jul 30 18:10:23.273051 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.273031    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-657b86df-serving-cert\""
Jul 30 18:10:23.286671 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.286647    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9"]
Jul 30 18:10:23.296872 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.296848    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:10:23.384557 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.384521    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grqhp\" (UniqueName: \"kubernetes.io/projected/5c40bea5-2943-4b9b-8f41-e336daacd4a5-kube-api-access-grqhp\") pod \"maas-api-e2e-mt-657b86df-5496595849-nfwf9\" (UID: \"5c40bea5-2943-4b9b-8f41-e336daacd4a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9"
Jul 30 18:10:23.384784 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.384656    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c40bea5-2943-4b9b-8f41-e336daacd4a5-maas-api-tls\") pod \"maas-api-e2e-mt-657b86df-5496595849-nfwf9\" (UID: \"5c40bea5-2943-4b9b-8f41-e336daacd4a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9"
Jul 30 18:10:23.384784 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.384706    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c40bea5-2943-4b9b-8f41-e336daacd4a5-openshift-service-ca\") pod \"maas-api-e2e-mt-657b86df-5496595849-nfwf9\" (UID: \"5c40bea5-2943-4b9b-8f41-e336daacd4a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9"
Jul 30 18:10:23.424407 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.424372    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:10:23.486053 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.486021    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c40bea5-2943-4b9b-8f41-e336daacd4a5-openshift-service-ca\") pod \"maas-api-e2e-mt-657b86df-5496595849-nfwf9\" (UID: \"5c40bea5-2943-4b9b-8f41-e336daacd4a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9"
Jul 30 18:10:23.486226 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.486098    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grqhp\" (UniqueName: \"kubernetes.io/projected/5c40bea5-2943-4b9b-8f41-e336daacd4a5-kube-api-access-grqhp\") pod \"maas-api-e2e-mt-657b86df-5496595849-nfwf9\" (UID: \"5c40bea5-2943-4b9b-8f41-e336daacd4a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9"
Jul 30 18:10:23.486226 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.486210    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c40bea5-2943-4b9b-8f41-e336daacd4a5-maas-api-tls\") pod \"maas-api-e2e-mt-657b86df-5496595849-nfwf9\" (UID: \"5c40bea5-2943-4b9b-8f41-e336daacd4a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9"
Jul 30 18:10:23.486757 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.486734    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c40bea5-2943-4b9b-8f41-e336daacd4a5-openshift-service-ca\") pod \"maas-api-e2e-mt-657b86df-5496595849-nfwf9\" (UID: \"5c40bea5-2943-4b9b-8f41-e336daacd4a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9"
Jul 30 18:10:23.488785 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.488763    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c40bea5-2943-4b9b-8f41-e336daacd4a5-maas-api-tls\") pod \"maas-api-e2e-mt-657b86df-5496595849-nfwf9\" (UID: \"5c40bea5-2943-4b9b-8f41-e336daacd4a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9"
Jul 30 18:10:23.496902 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.496881    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grqhp\" (UniqueName: \"kubernetes.io/projected/5c40bea5-2943-4b9b-8f41-e336daacd4a5-kube-api-access-grqhp\") pod \"maas-api-e2e-mt-657b86df-5496595849-nfwf9\" (UID: \"5c40bea5-2943-4b9b-8f41-e336daacd4a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9"
Jul 30 18:10:23.581797 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.581709    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9"
Jul 30 18:10:23.815200 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.815159    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:10:23.914322 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.914203    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:10:23.920873 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.920849    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs"]
Jul 30 18:10:23.924677 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.924648    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs"
Jul 30 18:10:23.927034 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.927016    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-657b86df\""
Jul 30 18:10:23.934267 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.934245    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs"]
Jul 30 18:10:23.940473 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.940454    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9"]
Jul 30 18:10:23.943478 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:10:23.943451    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c40bea5_2943_4b9b_8f41_e336daacd4a5.slice/crio-d378b07a72faf31cc75e91e5cad6aa53a3d36964b7c6f7ddfde021a0b0b4c021 WatchSource:0}: Error finding container d378b07a72faf31cc75e91e5cad6aa53a3d36964b7c6f7ddfde021a0b0b4c021: Status 404 returned error can't find the container with id d378b07a72faf31cc75e91e5cad6aa53a3d36964b7c6f7ddfde021a0b0b4c021
Jul 30 18:10:23.945275 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.945255    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 18:10:23.969842 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.969791    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth"]
Jul 30 18:10:23.973414 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.973391    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth"
Jul 30 18:10:23.975808 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.975785    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-657b86df-dockercfg-npr8r\""
Jul 30 18:10:23.985393 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.985370    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth"]
Jul 30 18:10:23.991478 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.991454    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcjn5\" (UniqueName: \"kubernetes.io/projected/cc1fdf2e-9712-4fe8-830c-3f066910d478-kube-api-access-qcjn5\") pod \"payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs\" (UID: \"cc1fdf2e-9712-4fe8-830c-3f066910d478\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs"
Jul 30 18:10:23.991581 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:23.991513    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc1fdf2e-9712-4fe8-830c-3f066910d478-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs\" (UID: \"cc1fdf2e-9712-4fe8-830c-3f066910d478\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs"
Jul 30 18:10:24.092510 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.092461    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcjn5\" (UniqueName: \"kubernetes.io/projected/cc1fdf2e-9712-4fe8-830c-3f066910d478-kube-api-access-qcjn5\") pod \"payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs\" (UID: \"cc1fdf2e-9712-4fe8-830c-3f066910d478\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs"
Jul 30 18:10:24.092695 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.092553    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14cedbd8-4198-4fab-8f3b-deee9015d26a-plugins-config-volume\") pod \"payload-processing-e2e-mt-657b86df-584d648b6b-6sdth\" (UID: \"14cedbd8-4198-4fab-8f3b-deee9015d26a\") " pod="openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth"
Jul 30 18:10:24.092695 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.092593    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc1fdf2e-9712-4fe8-830c-3f066910d478-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs\" (UID: \"cc1fdf2e-9712-4fe8-830c-3f066910d478\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs"
Jul 30 18:10:24.092695 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.092624    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnzrz\" (UniqueName: \"kubernetes.io/projected/14cedbd8-4198-4fab-8f3b-deee9015d26a-kube-api-access-fnzrz\") pod \"payload-processing-e2e-mt-657b86df-584d648b6b-6sdth\" (UID: \"14cedbd8-4198-4fab-8f3b-deee9015d26a\") " pod="openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth"
Jul 30 18:10:24.093360 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.093342    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc1fdf2e-9712-4fe8-830c-3f066910d478-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs\" (UID: \"cc1fdf2e-9712-4fe8-830c-3f066910d478\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs"
Jul 30 18:10:24.100926 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.100902    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcjn5\" (UniqueName: \"kubernetes.io/projected/cc1fdf2e-9712-4fe8-830c-3f066910d478-kube-api-access-qcjn5\") pod \"payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs\" (UID: \"cc1fdf2e-9712-4fe8-830c-3f066910d478\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs"
Jul 30 18:10:24.194180 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.194091    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14cedbd8-4198-4fab-8f3b-deee9015d26a-plugins-config-volume\") pod \"payload-processing-e2e-mt-657b86df-584d648b6b-6sdth\" (UID: \"14cedbd8-4198-4fab-8f3b-deee9015d26a\") " pod="openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth"
Jul 30 18:10:24.194180 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.194134    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnzrz\" (UniqueName: \"kubernetes.io/projected/14cedbd8-4198-4fab-8f3b-deee9015d26a-kube-api-access-fnzrz\") pod \"payload-processing-e2e-mt-657b86df-584d648b6b-6sdth\" (UID: \"14cedbd8-4198-4fab-8f3b-deee9015d26a\") " pod="openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth"
Jul 30 18:10:24.194784 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.194763    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14cedbd8-4198-4fab-8f3b-deee9015d26a-plugins-config-volume\") pod \"payload-processing-e2e-mt-657b86df-584d648b6b-6sdth\" (UID: \"14cedbd8-4198-4fab-8f3b-deee9015d26a\") " pod="openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth"
Jul 30 18:10:24.202766 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.202737    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnzrz\" (UniqueName: \"kubernetes.io/projected/14cedbd8-4198-4fab-8f3b-deee9015d26a-kube-api-access-fnzrz\") pod \"payload-processing-e2e-mt-657b86df-584d648b6b-6sdth\" (UID: \"14cedbd8-4198-4fab-8f3b-deee9015d26a\") " pod="openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth"
Jul 30 18:10:24.236652 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.236627    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs"
Jul 30 18:10:24.284565 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.284533    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth"
Jul 30 18:10:24.387942 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.387912    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs"]
Jul 30 18:10:24.389666 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:10:24.389638    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc1fdf2e_9712_4fe8_830c_3f066910d478.slice/crio-26a3fb33edd0bec49f4ff8cef2b93edf1a4d26b6141c270fb44812438ffca1d1 WatchSource:0}: Error finding container 26a3fb33edd0bec49f4ff8cef2b93edf1a4d26b6141c270fb44812438ffca1d1: Status 404 returned error can't find the container with id 26a3fb33edd0bec49f4ff8cef2b93edf1a4d26b6141c270fb44812438ffca1d1
Jul 30 18:10:24.448589 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.448563    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth"]
Jul 30 18:10:24.450416 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:10:24.450386    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14cedbd8_4198_4fab_8f3b_deee9015d26a.slice/crio-d981b7893880dcd7ecedf2f4134f13bfe8e275ae7c68a76817773eb89ca5a14b WatchSource:0}: Error finding container d981b7893880dcd7ecedf2f4134f13bfe8e275ae7c68a76817773eb89ca5a14b: Status 404 returned error can't find the container with id d981b7893880dcd7ecedf2f4134f13bfe8e275ae7c68a76817773eb89ca5a14b
Jul 30 18:10:24.713345 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.713253    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth" event={"ID":"14cedbd8-4198-4fab-8f3b-deee9015d26a","Type":"ContainerStarted","Data":"7a4abd48caa8d7816da2940a00f66e6329071053d793e019e5bb2691ab465759"}
Jul 30 18:10:24.713345 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.713312    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth" event={"ID":"14cedbd8-4198-4fab-8f3b-deee9015d26a","Type":"ContainerStarted","Data":"d981b7893880dcd7ecedf2f4134f13bfe8e275ae7c68a76817773eb89ca5a14b"}
Jul 30 18:10:24.713548 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.713368    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth"
Jul 30 18:10:24.714891 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.714859    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs" event={"ID":"cc1fdf2e-9712-4fe8-830c-3f066910d478","Type":"ContainerStarted","Data":"fb4dcf884c182559d269a0ac310e6dc1fdf33d81309add981e2a6bc69bb86dd5"}
Jul 30 18:10:24.715047 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.714906    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs" event={"ID":"cc1fdf2e-9712-4fe8-830c-3f066910d478","Type":"ContainerStarted","Data":"26a3fb33edd0bec49f4ff8cef2b93edf1a4d26b6141c270fb44812438ffca1d1"}
Jul 30 18:10:24.715047 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.714945    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs"
Jul 30 18:10:24.718051 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.717398    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9" event={"ID":"5c40bea5-2943-4b9b-8f41-e336daacd4a5","Type":"ContainerStarted","Data":"1b73feb6e9b66f4e4d2c623c5f8de0e17f40f3d855e0a39bf191ef7682635613"}
Jul 30 18:10:24.718051 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.717501    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9" event={"ID":"5c40bea5-2943-4b9b-8f41-e336daacd4a5","Type":"ContainerStarted","Data":"d378b07a72faf31cc75e91e5cad6aa53a3d36964b7c6f7ddfde021a0b0b4c021"}
Jul 30 18:10:24.718249 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.718163    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9"
Jul 30 18:10:24.731854 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.731789    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth" podStartSLOduration=1.7317762559999998 podStartE2EDuration="1.731776256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:10:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:10:24.730052355 +0000 UTC m=+2599.183616347" watchObservedRunningTime="2026-07-30 18:10:24.731776256 +0000 UTC m=+2599.185340253"
Jul 30 18:10:24.746569 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.746525    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs" podStartSLOduration=1.7465123409999999 podStartE2EDuration="1.746512341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:10:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:10:24.744584296 +0000 UTC m=+2599.198148290" watchObservedRunningTime="2026-07-30 18:10:24.746512341 +0000 UTC m=+2599.200076332"
Jul 30 18:10:24.763217 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:24.763169    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9" podStartSLOduration=1.307681675 podStartE2EDuration="1.763150333s" totalImagesPullingTime="455.468658ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:10:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:10:24.760315066 +0000 UTC m=+2599.213879057" watchObservedRunningTime="2026-07-30 18:10:24.763150333 +0000 UTC m=+2599.216714323"
Jul 30 18:10:30.731779 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:30.731748    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9"
Jul 30 18:10:35.727210 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:35.727177    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth"
Jul 30 18:10:35.727675 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:35.727349    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs"
Jul 30 18:10:47.221930 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:47.221894    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:10:48.180668 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:48.180620    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:10:49.534073 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:10:49.534035    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:11:37.493115 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.493081    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9"]
Jul 30 18:11:37.493668 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.493335    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9" podUID="5c40bea5-2943-4b9b-8f41-e336daacd4a5" containerName="maas-api" containerID="cri-o://1b73feb6e9b66f4e4d2c623c5f8de0e17f40f3d855e0a39bf191ef7682635613" gracePeriod=30
Jul 30 18:11:37.537973 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.537939    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:11:37.607045 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.607005    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth"]
Jul 30 18:11:37.607270 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.607247    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth" podUID="14cedbd8-4198-4fab-8f3b-deee9015d26a" containerName="payload-processing" containerID="cri-o://7a4abd48caa8d7816da2940a00f66e6329071053d793e019e5bb2691ab465759" gracePeriod=30
Jul 30 18:11:37.625225 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.625191    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs"]
Jul 30 18:11:37.625432 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.625410    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs" podUID="cc1fdf2e-9712-4fe8-830c-3f066910d478" containerName="payload-pre-processing" containerID="cri-o://fb4dcf884c182559d269a0ac310e6dc1fdf33d81309add981e2a6bc69bb86dd5" gracePeriod=30
Jul 30 18:11:37.745807 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.745764    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:11:37.777765 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.777733    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9"
Jul 30 18:11:37.795510 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.795481    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c40bea5-2943-4b9b-8f41-e336daacd4a5-maas-api-tls\") pod \"5c40bea5-2943-4b9b-8f41-e336daacd4a5\" (UID: \"5c40bea5-2943-4b9b-8f41-e336daacd4a5\") "
Jul 30 18:11:37.795683 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.795524    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c40bea5-2943-4b9b-8f41-e336daacd4a5-openshift-service-ca\") pod \"5c40bea5-2943-4b9b-8f41-e336daacd4a5\" (UID: \"5c40bea5-2943-4b9b-8f41-e336daacd4a5\") "
Jul 30 18:11:37.795683 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.795622    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grqhp\" (UniqueName: \"kubernetes.io/projected/5c40bea5-2943-4b9b-8f41-e336daacd4a5-kube-api-access-grqhp\") pod \"5c40bea5-2943-4b9b-8f41-e336daacd4a5\" (UID: \"5c40bea5-2943-4b9b-8f41-e336daacd4a5\") "
Jul 30 18:11:37.796315 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.796281    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c40bea5-2943-4b9b-8f41-e336daacd4a5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5c40bea5-2943-4b9b-8f41-e336daacd4a5" (UID: "5c40bea5-2943-4b9b-8f41-e336daacd4a5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:11:37.798510 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.798477    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c40bea5-2943-4b9b-8f41-e336daacd4a5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5c40bea5-2943-4b9b-8f41-e336daacd4a5" (UID: "5c40bea5-2943-4b9b-8f41-e336daacd4a5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 18:11:37.798510 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.798488    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c40bea5-2943-4b9b-8f41-e336daacd4a5-kube-api-access-grqhp" (OuterVolumeSpecName: "kube-api-access-grqhp") pod "5c40bea5-2943-4b9b-8f41-e336daacd4a5" (UID: "5c40bea5-2943-4b9b-8f41-e336daacd4a5"). InnerVolumeSpecName "kube-api-access-grqhp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:11:37.846447 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.846414    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:11:37.878414 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.878387    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth"
Jul 30 18:11:37.896605 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.896571    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnzrz\" (UniqueName: \"kubernetes.io/projected/14cedbd8-4198-4fab-8f3b-deee9015d26a-kube-api-access-fnzrz\") pod \"14cedbd8-4198-4fab-8f3b-deee9015d26a\" (UID: \"14cedbd8-4198-4fab-8f3b-deee9015d26a\") "
Jul 30 18:11:37.896793 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.896631    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14cedbd8-4198-4fab-8f3b-deee9015d26a-plugins-config-volume\") pod \"14cedbd8-4198-4fab-8f3b-deee9015d26a\" (UID: \"14cedbd8-4198-4fab-8f3b-deee9015d26a\") "
Jul 30 18:11:37.896974 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.896953    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grqhp\" (UniqueName: \"kubernetes.io/projected/5c40bea5-2943-4b9b-8f41-e336daacd4a5-kube-api-access-grqhp\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:11:37.897059 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.896980    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c40bea5-2943-4b9b-8f41-e336daacd4a5-maas-api-tls\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:11:37.897059 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.896995    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c40bea5-2943-4b9b-8f41-e336daacd4a5-openshift-service-ca\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:11:37.897129 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.897077    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14cedbd8-4198-4fab-8f3b-deee9015d26a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "14cedbd8-4198-4fab-8f3b-deee9015d26a" (UID: "14cedbd8-4198-4fab-8f3b-deee9015d26a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:11:37.899635 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.899607    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14cedbd8-4198-4fab-8f3b-deee9015d26a-kube-api-access-fnzrz" (OuterVolumeSpecName: "kube-api-access-fnzrz") pod "14cedbd8-4198-4fab-8f3b-deee9015d26a" (UID: "14cedbd8-4198-4fab-8f3b-deee9015d26a"). InnerVolumeSpecName "kube-api-access-fnzrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:11:37.917438 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.917398    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs"
Jul 30 18:11:37.928326 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.928300    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:11:37.998081 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.998050    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcjn5\" (UniqueName: \"kubernetes.io/projected/cc1fdf2e-9712-4fe8-830c-3f066910d478-kube-api-access-qcjn5\") pod \"cc1fdf2e-9712-4fe8-830c-3f066910d478\" (UID: \"cc1fdf2e-9712-4fe8-830c-3f066910d478\") "
Jul 30 18:11:37.998244 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.998165    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc1fdf2e-9712-4fe8-830c-3f066910d478-plugins-config-volume\") pod \"cc1fdf2e-9712-4fe8-830c-3f066910d478\" (UID: \"cc1fdf2e-9712-4fe8-830c-3f066910d478\") "
Jul 30 18:11:37.998338 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.998327    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnzrz\" (UniqueName: \"kubernetes.io/projected/14cedbd8-4198-4fab-8f3b-deee9015d26a-kube-api-access-fnzrz\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:11:37.998381 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.998341    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14cedbd8-4198-4fab-8f3b-deee9015d26a-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:11:37.998563 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:37.998539    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc1fdf2e-9712-4fe8-830c-3f066910d478-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cc1fdf2e-9712-4fe8-830c-3f066910d478" (UID: "cc1fdf2e-9712-4fe8-830c-3f066910d478"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:11:38.000375 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.000355    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc1fdf2e-9712-4fe8-830c-3f066910d478-kube-api-access-qcjn5" (OuterVolumeSpecName: "kube-api-access-qcjn5") pod "cc1fdf2e-9712-4fe8-830c-3f066910d478" (UID: "cc1fdf2e-9712-4fe8-830c-3f066910d478"). InnerVolumeSpecName "kube-api-access-qcjn5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:11:38.099107 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.099064    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc1fdf2e-9712-4fe8-830c-3f066910d478-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:11:38.099107 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.099101    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcjn5\" (UniqueName: \"kubernetes.io/projected/cc1fdf2e-9712-4fe8-830c-3f066910d478-kube-api-access-qcjn5\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:11:38.131416 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.131388    2577 generic.go:356] "Generic (PLEG): container finished" podID="14cedbd8-4198-4fab-8f3b-deee9015d26a" containerID="7a4abd48caa8d7816da2940a00f66e6329071053d793e019e5bb2691ab465759" exitCode=0
Jul 30 18:11:38.131543 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.131442    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth"
Jul 30 18:11:38.131543 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.131466    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth" event={"ID":"14cedbd8-4198-4fab-8f3b-deee9015d26a","Type":"ContainerDied","Data":"7a4abd48caa8d7816da2940a00f66e6329071053d793e019e5bb2691ab465759"}
Jul 30 18:11:38.131642 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.131546    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth" event={"ID":"14cedbd8-4198-4fab-8f3b-deee9015d26a","Type":"ContainerDied","Data":"d981b7893880dcd7ecedf2f4134f13bfe8e275ae7c68a76817773eb89ca5a14b"}
Jul 30 18:11:38.131642 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.131571    2577 scope.go:117] "RemoveContainer" containerID="7a4abd48caa8d7816da2940a00f66e6329071053d793e019e5bb2691ab465759"
Jul 30 18:11:38.132841 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.132798    2577 generic.go:356] "Generic (PLEG): container finished" podID="cc1fdf2e-9712-4fe8-830c-3f066910d478" containerID="fb4dcf884c182559d269a0ac310e6dc1fdf33d81309add981e2a6bc69bb86dd5" exitCode=0
Jul 30 18:11:38.132956 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.132863    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs"
Jul 30 18:11:38.132956 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.132895    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs" event={"ID":"cc1fdf2e-9712-4fe8-830c-3f066910d478","Type":"ContainerDied","Data":"fb4dcf884c182559d269a0ac310e6dc1fdf33d81309add981e2a6bc69bb86dd5"}
Jul 30 18:11:38.132956 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.132951    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs" event={"ID":"cc1fdf2e-9712-4fe8-830c-3f066910d478","Type":"ContainerDied","Data":"26a3fb33edd0bec49f4ff8cef2b93edf1a4d26b6141c270fb44812438ffca1d1"}
Jul 30 18:11:38.134427 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.134406    2577 generic.go:356] "Generic (PLEG): container finished" podID="5c40bea5-2943-4b9b-8f41-e336daacd4a5" containerID="1b73feb6e9b66f4e4d2c623c5f8de0e17f40f3d855e0a39bf191ef7682635613" exitCode=0
Jul 30 18:11:38.134534 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.134455    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9"
Jul 30 18:11:38.134534 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.134481    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9" event={"ID":"5c40bea5-2943-4b9b-8f41-e336daacd4a5","Type":"ContainerDied","Data":"1b73feb6e9b66f4e4d2c623c5f8de0e17f40f3d855e0a39bf191ef7682635613"}
Jul 30 18:11:38.134534 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.134524    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9" event={"ID":"5c40bea5-2943-4b9b-8f41-e336daacd4a5","Type":"ContainerDied","Data":"d378b07a72faf31cc75e91e5cad6aa53a3d36964b7c6f7ddfde021a0b0b4c021"}
Jul 30 18:11:38.141900 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.141882    2577 scope.go:117] "RemoveContainer" containerID="7a4abd48caa8d7816da2940a00f66e6329071053d793e019e5bb2691ab465759"
Jul 30 18:11:38.142222 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:11:38.142201    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a4abd48caa8d7816da2940a00f66e6329071053d793e019e5bb2691ab465759\": container with ID starting with 7a4abd48caa8d7816da2940a00f66e6329071053d793e019e5bb2691ab465759 not found: ID does not exist" containerID="7a4abd48caa8d7816da2940a00f66e6329071053d793e019e5bb2691ab465759"
Jul 30 18:11:38.142273 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.142232    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a4abd48caa8d7816da2940a00f66e6329071053d793e019e5bb2691ab465759"} err="failed to get container status \"7a4abd48caa8d7816da2940a00f66e6329071053d793e019e5bb2691ab465759\": rpc error: code = NotFound desc = could not find container \"7a4abd48caa8d7816da2940a00f66e6329071053d793e019e5bb2691ab465759\": container with ID starting with 7a4abd48caa8d7816da2940a00f66e6329071053d793e019e5bb2691ab465759 not found: ID does not exist"
Jul 30 18:11:38.142273 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.142251    2577 scope.go:117] "RemoveContainer" containerID="fb4dcf884c182559d269a0ac310e6dc1fdf33d81309add981e2a6bc69bb86dd5"
Jul 30 18:11:38.151384 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.151292    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth"]
Jul 30 18:11:38.151470 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.151411    2577 scope.go:117] "RemoveContainer" containerID="fb4dcf884c182559d269a0ac310e6dc1fdf33d81309add981e2a6bc69bb86dd5"
Jul 30 18:11:38.151762 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:11:38.151743    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb4dcf884c182559d269a0ac310e6dc1fdf33d81309add981e2a6bc69bb86dd5\": container with ID starting with fb4dcf884c182559d269a0ac310e6dc1fdf33d81309add981e2a6bc69bb86dd5 not found: ID does not exist" containerID="fb4dcf884c182559d269a0ac310e6dc1fdf33d81309add981e2a6bc69bb86dd5"
Jul 30 18:11:38.151854 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.151772    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb4dcf884c182559d269a0ac310e6dc1fdf33d81309add981e2a6bc69bb86dd5"} err="failed to get container status \"fb4dcf884c182559d269a0ac310e6dc1fdf33d81309add981e2a6bc69bb86dd5\": rpc error: code = NotFound desc = could not find container \"fb4dcf884c182559d269a0ac310e6dc1fdf33d81309add981e2a6bc69bb86dd5\": container with ID starting with fb4dcf884c182559d269a0ac310e6dc1fdf33d81309add981e2a6bc69bb86dd5 not found: ID does not exist"
Jul 30 18:11:38.151854 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.151798    2577 scope.go:117] "RemoveContainer" containerID="1b73feb6e9b66f4e4d2c623c5f8de0e17f40f3d855e0a39bf191ef7682635613"
Jul 30 18:11:38.158092 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.158066    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-657b86df-584d648b6b-6sdth"]
Jul 30 18:11:38.162235 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.162209    2577 scope.go:117] "RemoveContainer" containerID="1b73feb6e9b66f4e4d2c623c5f8de0e17f40f3d855e0a39bf191ef7682635613"
Jul 30 18:11:38.162636 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:11:38.162611    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b73feb6e9b66f4e4d2c623c5f8de0e17f40f3d855e0a39bf191ef7682635613\": container with ID starting with 1b73feb6e9b66f4e4d2c623c5f8de0e17f40f3d855e0a39bf191ef7682635613 not found: ID does not exist" containerID="1b73feb6e9b66f4e4d2c623c5f8de0e17f40f3d855e0a39bf191ef7682635613"
Jul 30 18:11:38.162730 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.162650    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b73feb6e9b66f4e4d2c623c5f8de0e17f40f3d855e0a39bf191ef7682635613"} err="failed to get container status \"1b73feb6e9b66f4e4d2c623c5f8de0e17f40f3d855e0a39bf191ef7682635613\": rpc error: code = NotFound desc = could not find container \"1b73feb6e9b66f4e4d2c623c5f8de0e17f40f3d855e0a39bf191ef7682635613\": container with ID starting with 1b73feb6e9b66f4e4d2c623c5f8de0e17f40f3d855e0a39bf191ef7682635613 not found: ID does not exist"
Jul 30 18:11:38.179209 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.179175    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs"]
Jul 30 18:11:38.181929 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.181903    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-657b86df-5957fc74d8-jz8gs"]
Jul 30 18:11:38.194532 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.194500    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9"]
Jul 30 18:11:38.198658 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:38.198624    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-657b86df-5496595849-nfwf9"]
Jul 30 18:11:40.111333 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:40.111300    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14cedbd8-4198-4fab-8f3b-deee9015d26a" path="/var/lib/kubelet/pods/14cedbd8-4198-4fab-8f3b-deee9015d26a/volumes"
Jul 30 18:11:40.111712 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:40.111679    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c40bea5-2943-4b9b-8f41-e336daacd4a5" path="/var/lib/kubelet/pods/5c40bea5-2943-4b9b-8f41-e336daacd4a5/volumes"
Jul 30 18:11:40.112020 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:40.112007    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc1fdf2e-9712-4fe8-830c-3f066910d478" path="/var/lib/kubelet/pods/cc1fdf2e-9712-4fe8-830c-3f066910d478/volumes"
Jul 30 18:11:41.266310 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:41.266282    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:11:46.293192 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:11:46.293162    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:12:23.285902 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:12:23.285868    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:12:24.485956 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:12:24.485917    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:12:29.379485 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:12:29.379452    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:12:29.479977 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:12:29.479927    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:12:29.663680 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:12:29.663579    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:12:29.965980 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:12:29.965869    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:12:51.539405 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:12:51.539365    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:12:51.628949 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:12:51.628916    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:12:51.735931 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:12:51.735899    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:12:52.368482 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:12:52.368433    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:12:52.465622 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:12:52.465586    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:12:52.563997 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:12:52.563964    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:12:53.865334 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:12:53.865301    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:12:59.650496 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:12:59.650461    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:12:59.829731 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:12:59.829700    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:13:00.257833 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:00.257785    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:13:00.357828 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:00.357786    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:13:00.462259 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:00.462226    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:13:06.213966 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:06.213932    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:13:06.298309 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:06.298276    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:13:06.382015 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:06.381982    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:13:08.995415 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:08.995375    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:13:16.969409 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:16.969372    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:13:17.235453 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:17.235374    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:13:38.261585 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:38.261533    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:13:44.833733 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:44.833683    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:13:44.933522 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:44.933458    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:13:45.245795 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:45.245769    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:13:45.594932 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:45.594807    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:13:45.962039 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:45.961959    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:13:48.285383 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:48.285337    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:13:48.377088 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:48.377044    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:13:49.364323 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:49.364285    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894"]
Jul 30 18:13:49.364980 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:49.364959    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc1fdf2e-9712-4fe8-830c-3f066910d478" containerName="payload-pre-processing"
Jul 30 18:13:49.364980 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:49.364979    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc1fdf2e-9712-4fe8-830c-3f066910d478" containerName="payload-pre-processing"
Jul 30 18:13:49.365161 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:49.365009    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14cedbd8-4198-4fab-8f3b-deee9015d26a" containerName="payload-processing"
Jul 30 18:13:49.365161 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:49.365019    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="14cedbd8-4198-4fab-8f3b-deee9015d26a" containerName="payload-processing"
Jul 30 18:13:49.365161 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:49.365028    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c40bea5-2943-4b9b-8f41-e336daacd4a5" containerName="maas-api"
Jul 30 18:13:49.365161 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:49.365033    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c40bea5-2943-4b9b-8f41-e336daacd4a5" containerName="maas-api"
Jul 30 18:13:49.365161 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:49.365120    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc1fdf2e-9712-4fe8-830c-3f066910d478" containerName="payload-pre-processing"
Jul 30 18:13:49.365161 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:49.365137    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="14cedbd8-4198-4fab-8f3b-deee9015d26a" containerName="payload-processing"
Jul 30 18:13:49.365161 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:49.365145    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c40bea5-2943-4b9b-8f41-e336daacd4a5" containerName="maas-api"
Jul 30 18:13:49.368507 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:49.368483    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894"
Jul 30 18:13:49.370655 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:49.370630    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-166ba9-serving-cert\""
Jul 30 18:13:49.377115 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:49.376511    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894"]
Jul 30 18:13:49.395227 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:49.395182    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:13:49.415502 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:49.415462    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1c68daf-7f4c-4c68-ad7d-355c26646347-maas-api-tls\") pod \"maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894\" (UID: \"d1c68daf-7f4c-4c68-ad7d-355c26646347\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894"
Jul 30 18:13:49.415502 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:49.415504    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxxfq\" (UniqueName: \"kubernetes.io/projected/d1c68daf-7f4c-4c68-ad7d-355c26646347-kube-api-access-pxxfq\") pod \"maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894\" (UID: \"d1c68daf-7f4c-4c68-ad7d-355c26646347\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894"
Jul 30 18:13:49.415729 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:49.415531    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1c68daf-7f4c-4c68-ad7d-355c26646347-openshift-service-ca\") pod \"maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894\" (UID: \"d1c68daf-7f4c-4c68-ad7d-355c26646347\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894"
Jul 30 18:13:49.516809 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:49.516767    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1c68daf-7f4c-4c68-ad7d-355c26646347-maas-api-tls\") pod \"maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894\" (UID: \"d1c68daf-7f4c-4c68-ad7d-355c26646347\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894"
Jul 30 18:13:49.517047 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:49.516843    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxxfq\" (UniqueName: \"kubernetes.io/projected/d1c68daf-7f4c-4c68-ad7d-355c26646347-kube-api-access-pxxfq\") pod \"maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894\" (UID: \"d1c68daf-7f4c-4c68-ad7d-355c26646347\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894"
Jul 30 18:13:49.517047 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:49.516885    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1c68daf-7f4c-4c68-ad7d-355c26646347-openshift-service-ca\") pod \"maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894\" (UID: \"d1c68daf-7f4c-4c68-ad7d-355c26646347\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894"
Jul 30 18:13:49.517047 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:13:49.516944    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-serving-cert: secret "maas-api-e2e-inf-a-166ba9-serving-cert" not found
Jul 30 18:13:49.517047 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:13:49.517022    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1c68daf-7f4c-4c68-ad7d-355c26646347-maas-api-tls podName:d1c68daf-7f4c-4c68-ad7d-355c26646347 nodeName:}" failed. No retries permitted until 2026-07-30 18:13:50.017002251 +0000 UTC m=+2804.470566227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d1c68daf-7f4c-4c68-ad7d-355c26646347-maas-api-tls") pod "maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894" (UID: "d1c68daf-7f4c-4c68-ad7d-355c26646347") : secret "maas-api-e2e-inf-a-166ba9-serving-cert" not found
Jul 30 18:13:49.517471 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:49.517454    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1c68daf-7f4c-4c68-ad7d-355c26646347-openshift-service-ca\") pod \"maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894\" (UID: \"d1c68daf-7f4c-4c68-ad7d-355c26646347\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894"
Jul 30 18:13:49.525441 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:49.525415    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxxfq\" (UniqueName: \"kubernetes.io/projected/d1c68daf-7f4c-4c68-ad7d-355c26646347-kube-api-access-pxxfq\") pod \"maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894\" (UID: \"d1c68daf-7f4c-4c68-ad7d-355c26646347\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894"
Jul 30 18:13:49.635054 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:49.634974    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:13:50.003693 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.003664    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:13:50.021013 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.020982    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1c68daf-7f4c-4c68-ad7d-355c26646347-maas-api-tls\") pod \"maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894\" (UID: \"d1c68daf-7f4c-4c68-ad7d-355c26646347\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894"
Jul 30 18:13:50.023680 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.023646    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1c68daf-7f4c-4c68-ad7d-355c26646347-maas-api-tls\") pod \"maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894\" (UID: \"d1c68daf-7f4c-4c68-ad7d-355c26646347\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894"
Jul 30 18:13:50.102800 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.102764    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:13:50.111265 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.111237    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk"]
Jul 30 18:13:50.115190 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.115172    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk"
Jul 30 18:13:50.117394 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.117372    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-166ba9-dockercfg-z7clw\""
Jul 30 18:13:50.117515 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.117396    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-166ba9\""
Jul 30 18:13:50.125209 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.125183    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk"]
Jul 30 18:13:50.145372 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.145343    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q"]
Jul 30 18:13:50.148768 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.148752    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q"
Jul 30 18:13:50.159405 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.159310    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q"]
Jul 30 18:13:50.223046 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.223010    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2de9067a-8554-48ed-888b-e99e6dca4679-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk\" (UID: \"2de9067a-8554-48ed-888b-e99e6dca4679\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk"
Jul 30 18:13:50.223240 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.223052    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q\" (UID: \"4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q"
Jul 30 18:13:50.223240 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.223162    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88hqc\" (UniqueName: \"kubernetes.io/projected/4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3-kube-api-access-88hqc\") pod \"payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q\" (UID: \"4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q"
Jul 30 18:13:50.223240 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.223208    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzkgp\" (UniqueName: \"kubernetes.io/projected/2de9067a-8554-48ed-888b-e99e6dca4679-kube-api-access-mzkgp\") pod \"payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk\" (UID: \"2de9067a-8554-48ed-888b-e99e6dca4679\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk"
Jul 30 18:13:50.281112 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.281035    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894"
Jul 30 18:13:50.324464 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.324428    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzkgp\" (UniqueName: \"kubernetes.io/projected/2de9067a-8554-48ed-888b-e99e6dca4679-kube-api-access-mzkgp\") pod \"payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk\" (UID: \"2de9067a-8554-48ed-888b-e99e6dca4679\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk"
Jul 30 18:13:50.324637 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.324512    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2de9067a-8554-48ed-888b-e99e6dca4679-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk\" (UID: \"2de9067a-8554-48ed-888b-e99e6dca4679\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk"
Jul 30 18:13:50.324637 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.324549    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q\" (UID: \"4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q"
Jul 30 18:13:50.324637 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.324628    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88hqc\" (UniqueName: \"kubernetes.io/projected/4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3-kube-api-access-88hqc\") pod \"payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q\" (UID: \"4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q"
Jul 30 18:13:50.325341 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.325316    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2de9067a-8554-48ed-888b-e99e6dca4679-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk\" (UID: \"2de9067a-8554-48ed-888b-e99e6dca4679\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk"
Jul 30 18:13:50.325450 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.325316    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q\" (UID: \"4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q"
Jul 30 18:13:50.334918 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.334884    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88hqc\" (UniqueName: \"kubernetes.io/projected/4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3-kube-api-access-88hqc\") pod \"payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q\" (UID: \"4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q"
Jul 30 18:13:50.335125 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.335059    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzkgp\" (UniqueName: \"kubernetes.io/projected/2de9067a-8554-48ed-888b-e99e6dca4679-kube-api-access-mzkgp\") pod \"payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk\" (UID: \"2de9067a-8554-48ed-888b-e99e6dca4679\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk"
Jul 30 18:13:50.422365 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.422339    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894"]
Jul 30 18:13:50.424980 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:13:50.424954    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1c68daf_7f4c_4c68_ad7d_355c26646347.slice/crio-8b8447e1d0a99a6a8145735a6293063cc732d8f5137381c8229d3cee7d51e5c8 WatchSource:0}: Error finding container 8b8447e1d0a99a6a8145735a6293063cc732d8f5137381c8229d3cee7d51e5c8: Status 404 returned error can't find the container with id 8b8447e1d0a99a6a8145735a6293063cc732d8f5137381c8229d3cee7d51e5c8
Jul 30 18:13:50.427925 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.427903    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk"
Jul 30 18:13:50.460269 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.460233    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q"
Jul 30 18:13:50.587981 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.587949    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk"]
Jul 30 18:13:50.589269 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:13:50.589236    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2de9067a_8554_48ed_888b_e99e6dca4679.slice/crio-fd10183bee7941e673a928d225635adb21ebb5d956852d1dae4969ba4ffe19f8 WatchSource:0}: Error finding container fd10183bee7941e673a928d225635adb21ebb5d956852d1dae4969ba4ffe19f8: Status 404 returned error can't find the container with id fd10183bee7941e673a928d225635adb21ebb5d956852d1dae4969ba4ffe19f8
Jul 30 18:13:50.626616 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.626582    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q"]
Jul 30 18:13:50.633156 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:13:50.633126    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4265e0c7_5b54_44d3_ac93_58bf0cf8b5c3.slice/crio-59518bb019ecfc6b4964beac0e325bd1c125ecf89e5c8d48d620cb74bff54f79 WatchSource:0}: Error finding container 59518bb019ecfc6b4964beac0e325bd1c125ecf89e5c8d48d620cb74bff54f79: Status 404 returned error can't find the container with id 59518bb019ecfc6b4964beac0e325bd1c125ecf89e5c8d48d620cb74bff54f79
Jul 30 18:13:50.839762 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.839667    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk" event={"ID":"2de9067a-8554-48ed-888b-e99e6dca4679","Type":"ContainerStarted","Data":"45cd59897160d959035b84ffa836231553e271e383279ca04b3ee8b268868283"}
Jul 30 18:13:50.839762 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.839723    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk" event={"ID":"2de9067a-8554-48ed-888b-e99e6dca4679","Type":"ContainerStarted","Data":"fd10183bee7941e673a928d225635adb21ebb5d956852d1dae4969ba4ffe19f8"}
Jul 30 18:13:50.840008 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.839789    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk"
Jul 30 18:13:50.841041 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.841017    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894" event={"ID":"d1c68daf-7f4c-4c68-ad7d-355c26646347","Type":"ContainerStarted","Data":"8b8447e1d0a99a6a8145735a6293063cc732d8f5137381c8229d3cee7d51e5c8"}
Jul 30 18:13:50.842493 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.842473    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q" event={"ID":"4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3","Type":"ContainerStarted","Data":"e41cb692a631324cfacb4b1eae63b29f184edd50713317468c126410854c2b87"}
Jul 30 18:13:50.842588 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.842508    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q" event={"ID":"4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3","Type":"ContainerStarted","Data":"59518bb019ecfc6b4964beac0e325bd1c125ecf89e5c8d48d620cb74bff54f79"}
Jul 30 18:13:50.842588 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.842549    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q"
Jul 30 18:13:50.858097 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.858040    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk" podStartSLOduration=0.858019495 podStartE2EDuration="858.019495ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:13:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:13:50.856374291 +0000 UTC m=+2805.309938284" watchObservedRunningTime="2026-07-30 18:13:50.858019495 +0000 UTC m=+2805.311583487"
Jul 30 18:13:50.872602 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:50.872546    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q" podStartSLOduration=0.872530408 podStartE2EDuration="872.530408ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:13:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:13:50.871726412 +0000 UTC m=+2805.325290405" watchObservedRunningTime="2026-07-30 18:13:50.872530408 +0000 UTC m=+2805.326094399"
Jul 30 18:13:51.849226 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:51.849176    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894" event={"ID":"d1c68daf-7f4c-4c68-ad7d-355c26646347","Type":"ContainerStarted","Data":"18abd8ad8b1059638c229a2d6dcc1643c42795239f06db5b3a2f5038d20bff81"}
Jul 30 18:13:51.849666 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:51.849623    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894"
Jul 30 18:13:51.866961 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:51.866894    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894" podStartSLOduration=2.299430677 podStartE2EDuration="2.866877533s" totalImagesPullingTime="567.446856ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:13:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:13:51.865596045 +0000 UTC m=+2806.319160037" watchObservedRunningTime="2026-07-30 18:13:51.866877533 +0000 UTC m=+2806.320441524"
Jul 30 18:13:58.865876 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:13:58.865811    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894"
Jul 30 18:14:01.852040 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:01.851995    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk"
Jul 30 18:14:01.852456 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:01.852184    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q"
Jul 30 18:14:13.350000 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:13.349962    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:14:13.706948 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:13.706852    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:14:14.038441 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:14.038409    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:14:14.411123 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:14.411042    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:14:17.420191 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:17.420149    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:14:17.520876 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:17.520810    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:14:18.522563 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:18.522529    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr"]
Jul 30 18:14:18.526896 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:18.526874    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr"
Jul 30 18:14:18.529074 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:18.529054    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-984898-serving-cert\""
Jul 30 18:14:18.535171 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:18.535141    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr"]
Jul 30 18:14:18.548688 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:18.548659    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:14:18.680195 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:18.680162    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:14:18.692799 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:18.692770    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a7af5d4-2f84-499b-9a05-e406653b25bc-openshift-service-ca\") pod \"maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr\" (UID: \"5a7af5d4-2f84-499b-9a05-e406653b25bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr"
Jul 30 18:14:18.692930 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:18.692801    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dj5b\" (UniqueName: \"kubernetes.io/projected/5a7af5d4-2f84-499b-9a05-e406653b25bc-kube-api-access-8dj5b\") pod \"maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr\" (UID: \"5a7af5d4-2f84-499b-9a05-e406653b25bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr"
Jul 30 18:14:18.692930 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:18.692893    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a7af5d4-2f84-499b-9a05-e406653b25bc-maas-api-tls\") pod \"maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr\" (UID: \"5a7af5d4-2f84-499b-9a05-e406653b25bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr"
Jul 30 18:14:18.794534 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:18.794436    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a7af5d4-2f84-499b-9a05-e406653b25bc-openshift-service-ca\") pod \"maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr\" (UID: \"5a7af5d4-2f84-499b-9a05-e406653b25bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr"
Jul 30 18:14:18.794534 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:18.794471    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dj5b\" (UniqueName: \"kubernetes.io/projected/5a7af5d4-2f84-499b-9a05-e406653b25bc-kube-api-access-8dj5b\") pod \"maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr\" (UID: \"5a7af5d4-2f84-499b-9a05-e406653b25bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr"
Jul 30 18:14:18.794534 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:18.794505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a7af5d4-2f84-499b-9a05-e406653b25bc-maas-api-tls\") pod \"maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr\" (UID: \"5a7af5d4-2f84-499b-9a05-e406653b25bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr"
Jul 30 18:14:18.802695 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:18.795691    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a7af5d4-2f84-499b-9a05-e406653b25bc-openshift-service-ca\") pod \"maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr\" (UID: \"5a7af5d4-2f84-499b-9a05-e406653b25bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr"
Jul 30 18:14:18.802695 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:18.798160    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a7af5d4-2f84-499b-9a05-e406653b25bc-maas-api-tls\") pod \"maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr\" (UID: \"5a7af5d4-2f84-499b-9a05-e406653b25bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr"
Jul 30 18:14:18.804104 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:18.804078    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dj5b\" (UniqueName: \"kubernetes.io/projected/5a7af5d4-2f84-499b-9a05-e406653b25bc-kube-api-access-8dj5b\") pod \"maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr\" (UID: \"5a7af5d4-2f84-499b-9a05-e406653b25bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr"
Jul 30 18:14:18.842059 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:18.842022    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr"
Jul 30 18:14:18.990117 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:18.990086    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr"]
Jul 30 18:14:18.991936 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:14:18.991906    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a7af5d4_2f84_499b_9a05_e406653b25bc.slice/crio-e435ce8b6848f44568c81c752afed3e2f5c8eff915621332adf3825648f3f720 WatchSource:0}: Error finding container e435ce8b6848f44568c81c752afed3e2f5c8eff915621332adf3825648f3f720: Status 404 returned error can't find the container with id e435ce8b6848f44568c81c752afed3e2f5c8eff915621332adf3825648f3f720
Jul 30 18:14:19.081932 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.081811    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:14:19.182248 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.182210    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:14:19.206399 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.206369    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z"]
Jul 30 18:14:19.211179 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.211162    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z"
Jul 30 18:14:19.213955 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.213936    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-984898\""
Jul 30 18:14:19.214077 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.213940    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-984898-dockercfg-6h4hl\""
Jul 30 18:14:19.220680 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.220655    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z"]
Jul 30 18:14:19.264045 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.264011    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl"]
Jul 30 18:14:19.267926 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.267908    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl"
Jul 30 18:14:19.282021 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.281998    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl"]
Jul 30 18:14:19.300330 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.300296    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2gv7\" (UniqueName: \"kubernetes.io/projected/8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7-kube-api-access-p2gv7\") pod \"payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z\" (UID: \"8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7\") " pod="openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z"
Jul 30 18:14:19.300475 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.300417    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z\" (UID: \"8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7\") " pod="openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z"
Jul 30 18:14:19.402057 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.401986    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z\" (UID: \"8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7\") " pod="openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z"
Jul 30 18:14:19.402057 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.402026    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q98t\" (UniqueName: \"kubernetes.io/projected/52bf434a-8de0-49bf-878f-be4e1c3651b1-kube-api-access-9q98t\") pod \"payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl\" (UID: \"52bf434a-8de0-49bf-878f-be4e1c3651b1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl"
Jul 30 18:14:19.402057 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.402049    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52bf434a-8de0-49bf-878f-be4e1c3651b1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl\" (UID: \"52bf434a-8de0-49bf-878f-be4e1c3651b1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl"
Jul 30 18:14:19.402324 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.402196    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2gv7\" (UniqueName: \"kubernetes.io/projected/8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7-kube-api-access-p2gv7\") pod \"payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z\" (UID: \"8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7\") " pod="openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z"
Jul 30 18:14:19.402930 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.402901    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z\" (UID: \"8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7\") " pod="openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z"
Jul 30 18:14:19.410595 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.410567    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2gv7\" (UniqueName: \"kubernetes.io/projected/8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7-kube-api-access-p2gv7\") pod \"payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z\" (UID: \"8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7\") " pod="openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z"
Jul 30 18:14:19.503376 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.503340    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q98t\" (UniqueName: \"kubernetes.io/projected/52bf434a-8de0-49bf-878f-be4e1c3651b1-kube-api-access-9q98t\") pod \"payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl\" (UID: \"52bf434a-8de0-49bf-878f-be4e1c3651b1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl"
Jul 30 18:14:19.503536 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.503382    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52bf434a-8de0-49bf-878f-be4e1c3651b1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl\" (UID: \"52bf434a-8de0-49bf-878f-be4e1c3651b1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl"
Jul 30 18:14:19.504135 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.504113    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52bf434a-8de0-49bf-878f-be4e1c3651b1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl\" (UID: \"52bf434a-8de0-49bf-878f-be4e1c3651b1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl"
Jul 30 18:14:19.513177 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.513152    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q98t\" (UniqueName: \"kubernetes.io/projected/52bf434a-8de0-49bf-878f-be4e1c3651b1-kube-api-access-9q98t\") pod \"payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl\" (UID: \"52bf434a-8de0-49bf-878f-be4e1c3651b1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl"
Jul 30 18:14:19.524019 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.523999    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z"
Jul 30 18:14:19.578293 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.578246    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl"
Jul 30 18:14:19.688248 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.688209    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z"]
Jul 30 18:14:19.689308 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:14:19.689281    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bfab8e5_3dd1_4581_a5ea_17a6edb2c2c7.slice/crio-36d4f94260581812bce824693c9bbf9658608e5952d8c7f6b8fc9fcf4b7e41dc WatchSource:0}: Error finding container 36d4f94260581812bce824693c9bbf9658608e5952d8c7f6b8fc9fcf4b7e41dc: Status 404 returned error can't find the container with id 36d4f94260581812bce824693c9bbf9658608e5952d8c7f6b8fc9fcf4b7e41dc
Jul 30 18:14:19.746238 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:19.746019    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl"]
Jul 30 18:14:19.746921 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:14:19.746896    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52bf434a_8de0_49bf_878f_be4e1c3651b1.slice/crio-c9522a132643831c01443804e6dfab9a249c88f7a04e8e22597c2939f6e3a914 WatchSource:0}: Error finding container c9522a132643831c01443804e6dfab9a249c88f7a04e8e22597c2939f6e3a914: Status 404 returned error can't find the container with id c9522a132643831c01443804e6dfab9a249c88f7a04e8e22597c2939f6e3a914
Jul 30 18:14:20.006587 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:20.006550    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl" event={"ID":"52bf434a-8de0-49bf-878f-be4e1c3651b1","Type":"ContainerStarted","Data":"c5788203d1509324ac7c50b0e35fdda5a194a9f10d18ff3758c16f29f9c26fb7"}
Jul 30 18:14:20.006792 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:20.006612    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl" event={"ID":"52bf434a-8de0-49bf-878f-be4e1c3651b1","Type":"ContainerStarted","Data":"c9522a132643831c01443804e6dfab9a249c88f7a04e8e22597c2939f6e3a914"}
Jul 30 18:14:20.006792 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:20.006643    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl"
Jul 30 18:14:20.008177 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:20.008154    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z" event={"ID":"8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7","Type":"ContainerStarted","Data":"a3a8aacfb81109d9931f466523dad0c28f2de8643b2b6ab3a2affadfb8524d82"}
Jul 30 18:14:20.008301 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:20.008193    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z" event={"ID":"8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7","Type":"ContainerStarted","Data":"36d4f94260581812bce824693c9bbf9658608e5952d8c7f6b8fc9fcf4b7e41dc"}
Jul 30 18:14:20.008301 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:20.008218    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z"
Jul 30 18:14:20.009604 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:20.009586    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr" event={"ID":"5a7af5d4-2f84-499b-9a05-e406653b25bc","Type":"ContainerStarted","Data":"e622baf52a72c93b67e7396a760361041c89227805bd399546a5228b1de3bfe8"}
Jul 30 18:14:20.009702 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:20.009626    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr" event={"ID":"5a7af5d4-2f84-499b-9a05-e406653b25bc","Type":"ContainerStarted","Data":"e435ce8b6848f44568c81c752afed3e2f5c8eff915621332adf3825648f3f720"}
Jul 30 18:14:20.009702 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:20.009681    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr"
Jul 30 18:14:20.025705 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:20.025662    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl" podStartSLOduration=1.025649481 podStartE2EDuration="1.025649481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:14:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:14:20.023161705 +0000 UTC m=+2834.476725695" watchObservedRunningTime="2026-07-30 18:14:20.025649481 +0000 UTC m=+2834.479213472"
Jul 30 18:14:20.042314 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:20.042273    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr" podStartSLOduration=1.625927159 podStartE2EDuration="2.042259616s" totalImagesPullingTime="416.332457ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:14:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:14:20.041140697 +0000 UTC m=+2834.494704715" watchObservedRunningTime="2026-07-30 18:14:20.042259616 +0000 UTC m=+2834.495823606"
Jul 30 18:14:20.064358 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:20.064289    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z" podStartSLOduration=1.064264418 podStartE2EDuration="1.064264418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:14:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:14:20.060446131 +0000 UTC m=+2834.514010124" watchObservedRunningTime="2026-07-30 18:14:20.064264418 +0000 UTC m=+2834.517828411"
Jul 30 18:14:26.020750 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:26.020715    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr"
Jul 30 18:14:31.016923 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:31.016885    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl"
Jul 30 18:14:31.017301 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:31.016962    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z"
Jul 30 18:14:42.491722 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:42.491690    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:14:49.549860 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:49.549798    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:14:54.643001 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:14:54.642948    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:15:21.346856 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:15:21.346807    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:15:54.111886 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:15:54.111835    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:02.678207 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:02.678150    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:03.606909 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:03.606877    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:14.172182 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.172144    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894"]
Jul 30 18:16:14.172951 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.172899    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894" podUID="d1c68daf-7f4c-4c68-ad7d-355c26646347" containerName="maas-api" containerID="cri-o://18abd8ad8b1059638c229a2d6dcc1643c42795239f06db5b3a2f5038d20bff81" gracePeriod=30
Jul 30 18:16:14.203590 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.203559    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:14.228561 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.228526    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q"]
Jul 30 18:16:14.228798 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.228759    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q" podUID="4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3" containerName="payload-processing" containerID="cri-o://e41cb692a631324cfacb4b1eae63b29f184edd50713317468c126410854c2b87" gracePeriod=30
Jul 30 18:16:14.234684 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.234654    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk"]
Jul 30 18:16:14.234941 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.234904    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk" podUID="2de9067a-8554-48ed-888b-e99e6dca4679" containerName="payload-pre-processing" containerID="cri-o://45cd59897160d959035b84ffa836231553e271e383279ca04b3ee8b268868283" gracePeriod=30
Jul 30 18:16:14.302673 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.302645    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:14.398228 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.398191    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:14.516379 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.516351    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894"
Jul 30 18:16:14.531910 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.531885    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q"
Jul 30 18:16:14.550031 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.549996    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk"
Jul 30 18:16:14.582732 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.582687    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:14.613840 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.613792    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2de9067a-8554-48ed-888b-e99e6dca4679-plugins-config-volume\") pod \"2de9067a-8554-48ed-888b-e99e6dca4679\" (UID: \"2de9067a-8554-48ed-888b-e99e6dca4679\") "
Jul 30 18:16:14.613840 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.613836    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzkgp\" (UniqueName: \"kubernetes.io/projected/2de9067a-8554-48ed-888b-e99e6dca4679-kube-api-access-mzkgp\") pod \"2de9067a-8554-48ed-888b-e99e6dca4679\" (UID: \"2de9067a-8554-48ed-888b-e99e6dca4679\") "
Jul 30 18:16:14.614070 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.613866    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1c68daf-7f4c-4c68-ad7d-355c26646347-openshift-service-ca\") pod \"d1c68daf-7f4c-4c68-ad7d-355c26646347\" (UID: \"d1c68daf-7f4c-4c68-ad7d-355c26646347\") "
Jul 30 18:16:14.614070 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.613920    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88hqc\" (UniqueName: \"kubernetes.io/projected/4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3-kube-api-access-88hqc\") pod \"4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3\" (UID: \"4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3\") "
Jul 30 18:16:14.614070 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.613988    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1c68daf-7f4c-4c68-ad7d-355c26646347-maas-api-tls\") pod \"d1c68daf-7f4c-4c68-ad7d-355c26646347\" (UID: \"d1c68daf-7f4c-4c68-ad7d-355c26646347\") "
Jul 30 18:16:14.614070 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.614004    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3-plugins-config-volume\") pod \"4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3\" (UID: \"4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3\") "
Jul 30 18:16:14.614070 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.614056    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxxfq\" (UniqueName: \"kubernetes.io/projected/d1c68daf-7f4c-4c68-ad7d-355c26646347-kube-api-access-pxxfq\") pod \"d1c68daf-7f4c-4c68-ad7d-355c26646347\" (UID: \"d1c68daf-7f4c-4c68-ad7d-355c26646347\") "
Jul 30 18:16:14.614372 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.614339    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1c68daf-7f4c-4c68-ad7d-355c26646347-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d1c68daf-7f4c-4c68-ad7d-355c26646347" (UID: "d1c68daf-7f4c-4c68-ad7d-355c26646347"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:16:14.614372 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.614368    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2de9067a-8554-48ed-888b-e99e6dca4679-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2de9067a-8554-48ed-888b-e99e6dca4679" (UID: "2de9067a-8554-48ed-888b-e99e6dca4679"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:16:14.614559 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.614448    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3" (UID: "4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:16:14.616320 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.616281    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3-kube-api-access-88hqc" (OuterVolumeSpecName: "kube-api-access-88hqc") pod "4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3" (UID: "4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3"). InnerVolumeSpecName "kube-api-access-88hqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:16:14.616422 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.616345    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2de9067a-8554-48ed-888b-e99e6dca4679-kube-api-access-mzkgp" (OuterVolumeSpecName: "kube-api-access-mzkgp") pod "2de9067a-8554-48ed-888b-e99e6dca4679" (UID: "2de9067a-8554-48ed-888b-e99e6dca4679"). InnerVolumeSpecName "kube-api-access-mzkgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:16:14.616422 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.616357    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1c68daf-7f4c-4c68-ad7d-355c26646347-kube-api-access-pxxfq" (OuterVolumeSpecName: "kube-api-access-pxxfq") pod "d1c68daf-7f4c-4c68-ad7d-355c26646347" (UID: "d1c68daf-7f4c-4c68-ad7d-355c26646347"). InnerVolumeSpecName "kube-api-access-pxxfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:16:14.616422 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.616388    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1c68daf-7f4c-4c68-ad7d-355c26646347-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d1c68daf-7f4c-4c68-ad7d-355c26646347" (UID: "d1c68daf-7f4c-4c68-ad7d-355c26646347"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 18:16:14.632521 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.632486    2577 generic.go:356] "Generic (PLEG): container finished" podID="2de9067a-8554-48ed-888b-e99e6dca4679" containerID="45cd59897160d959035b84ffa836231553e271e383279ca04b3ee8b268868283" exitCode=0
Jul 30 18:16:14.632688 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.632547    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk"
Jul 30 18:16:14.632688 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.632566    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk" event={"ID":"2de9067a-8554-48ed-888b-e99e6dca4679","Type":"ContainerDied","Data":"45cd59897160d959035b84ffa836231553e271e383279ca04b3ee8b268868283"}
Jul 30 18:16:14.632688 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.632648    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk" event={"ID":"2de9067a-8554-48ed-888b-e99e6dca4679","Type":"ContainerDied","Data":"fd10183bee7941e673a928d225635adb21ebb5d956852d1dae4969ba4ffe19f8"}
Jul 30 18:16:14.632865 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.632667    2577 scope.go:117] "RemoveContainer" containerID="45cd59897160d959035b84ffa836231553e271e383279ca04b3ee8b268868283"
Jul 30 18:16:14.633981 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.633962    2577 generic.go:356] "Generic (PLEG): container finished" podID="d1c68daf-7f4c-4c68-ad7d-355c26646347" containerID="18abd8ad8b1059638c229a2d6dcc1643c42795239f06db5b3a2f5038d20bff81" exitCode=0
Jul 30 18:16:14.634073 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.634016    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894"
Jul 30 18:16:14.634073 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.634021    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894" event={"ID":"d1c68daf-7f4c-4c68-ad7d-355c26646347","Type":"ContainerDied","Data":"18abd8ad8b1059638c229a2d6dcc1643c42795239f06db5b3a2f5038d20bff81"}
Jul 30 18:16:14.634174 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.634072    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894" event={"ID":"d1c68daf-7f4c-4c68-ad7d-355c26646347","Type":"ContainerDied","Data":"8b8447e1d0a99a6a8145735a6293063cc732d8f5137381c8229d3cee7d51e5c8"}
Jul 30 18:16:14.635484 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.635458    2577 generic.go:356] "Generic (PLEG): container finished" podID="4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3" containerID="e41cb692a631324cfacb4b1eae63b29f184edd50713317468c126410854c2b87" exitCode=0
Jul 30 18:16:14.635618 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.635520    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q"
Jul 30 18:16:14.635700 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.635661    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q" event={"ID":"4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3","Type":"ContainerDied","Data":"e41cb692a631324cfacb4b1eae63b29f184edd50713317468c126410854c2b87"}
Jul 30 18:16:14.635763 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.635709    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q" event={"ID":"4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3","Type":"ContainerDied","Data":"59518bb019ecfc6b4964beac0e325bd1c125ecf89e5c8d48d620cb74bff54f79"}
Jul 30 18:16:14.642790 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.642765    2577 scope.go:117] "RemoveContainer" containerID="45cd59897160d959035b84ffa836231553e271e383279ca04b3ee8b268868283"
Jul 30 18:16:14.643243 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:16:14.643203    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45cd59897160d959035b84ffa836231553e271e383279ca04b3ee8b268868283\": container with ID starting with 45cd59897160d959035b84ffa836231553e271e383279ca04b3ee8b268868283 not found: ID does not exist" containerID="45cd59897160d959035b84ffa836231553e271e383279ca04b3ee8b268868283"
Jul 30 18:16:14.643334 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.643260    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45cd59897160d959035b84ffa836231553e271e383279ca04b3ee8b268868283"} err="failed to get container status \"45cd59897160d959035b84ffa836231553e271e383279ca04b3ee8b268868283\": rpc error: code = NotFound desc = could not find container \"45cd59897160d959035b84ffa836231553e271e383279ca04b3ee8b268868283\": container with ID starting with 45cd59897160d959035b84ffa836231553e271e383279ca04b3ee8b268868283 not found: ID does not exist"
Jul 30 18:16:14.643334 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.643285    2577 scope.go:117] "RemoveContainer" containerID="18abd8ad8b1059638c229a2d6dcc1643c42795239f06db5b3a2f5038d20bff81"
Jul 30 18:16:14.653882 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.653865    2577 scope.go:117] "RemoveContainer" containerID="18abd8ad8b1059638c229a2d6dcc1643c42795239f06db5b3a2f5038d20bff81"
Jul 30 18:16:14.654166 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:16:14.654146    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18abd8ad8b1059638c229a2d6dcc1643c42795239f06db5b3a2f5038d20bff81\": container with ID starting with 18abd8ad8b1059638c229a2d6dcc1643c42795239f06db5b3a2f5038d20bff81 not found: ID does not exist" containerID="18abd8ad8b1059638c229a2d6dcc1643c42795239f06db5b3a2f5038d20bff81"
Jul 30 18:16:14.654241 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.654171    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18abd8ad8b1059638c229a2d6dcc1643c42795239f06db5b3a2f5038d20bff81"} err="failed to get container status \"18abd8ad8b1059638c229a2d6dcc1643c42795239f06db5b3a2f5038d20bff81\": rpc error: code = NotFound desc = could not find container \"18abd8ad8b1059638c229a2d6dcc1643c42795239f06db5b3a2f5038d20bff81\": container with ID starting with 18abd8ad8b1059638c229a2d6dcc1643c42795239f06db5b3a2f5038d20bff81 not found: ID does not exist"
Jul 30 18:16:14.654241 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.654193    2577 scope.go:117] "RemoveContainer" containerID="e41cb692a631324cfacb4b1eae63b29f184edd50713317468c126410854c2b87"
Jul 30 18:16:14.660473 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.660450    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk"]
Jul 30 18:16:14.662604 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.662586    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-166ba9-654968b888-gw6pk"]
Jul 30 18:16:14.664363 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.664346    2577 scope.go:117] "RemoveContainer" containerID="e41cb692a631324cfacb4b1eae63b29f184edd50713317468c126410854c2b87"
Jul 30 18:16:14.664703 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:16:14.664680    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e41cb692a631324cfacb4b1eae63b29f184edd50713317468c126410854c2b87\": container with ID starting with e41cb692a631324cfacb4b1eae63b29f184edd50713317468c126410854c2b87 not found: ID does not exist" containerID="e41cb692a631324cfacb4b1eae63b29f184edd50713317468c126410854c2b87"
Jul 30 18:16:14.664797 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.664713    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e41cb692a631324cfacb4b1eae63b29f184edd50713317468c126410854c2b87"} err="failed to get container status \"e41cb692a631324cfacb4b1eae63b29f184edd50713317468c126410854c2b87\": rpc error: code = NotFound desc = could not find container \"e41cb692a631324cfacb4b1eae63b29f184edd50713317468c126410854c2b87\": container with ID starting with e41cb692a631324cfacb4b1eae63b29f184edd50713317468c126410854c2b87 not found: ID does not exist"
Jul 30 18:16:14.671230 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.671208    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894"]
Jul 30 18:16:14.675046 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.675024    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-166ba9-5d99dc95f4-cc894"]
Jul 30 18:16:14.685058 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.685028    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q"]
Jul 30 18:16:14.687092 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.687071    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-166ba9-75cdd7f964-smb9q"]
Jul 30 18:16:14.715610 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.715522    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1c68daf-7f4c-4c68-ad7d-355c26646347-openshift-service-ca\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:16:14.715610 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.715556    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88hqc\" (UniqueName: \"kubernetes.io/projected/4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3-kube-api-access-88hqc\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:16:14.715610 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.715575    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1c68daf-7f4c-4c68-ad7d-355c26646347-maas-api-tls\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:16:14.715610 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.715587    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:16:14.715610 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.715600    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxxfq\" (UniqueName: \"kubernetes.io/projected/d1c68daf-7f4c-4c68-ad7d-355c26646347-kube-api-access-pxxfq\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:16:14.715610 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.715613    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2de9067a-8554-48ed-888b-e99e6dca4679-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:16:14.715900 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.715626    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzkgp\" (UniqueName: \"kubernetes.io/projected/2de9067a-8554-48ed-888b-e99e6dca4679-kube-api-access-mzkgp\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:16:14.806249 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:14.806212    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:16.112704 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:16.112666    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2de9067a-8554-48ed-888b-e99e6dca4679" path="/var/lib/kubelet/pods/2de9067a-8554-48ed-888b-e99e6dca4679/volumes"
Jul 30 18:16:16.113250 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:16.113230    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3" path="/var/lib/kubelet/pods/4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3/volumes"
Jul 30 18:16:16.113654 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:16.113635    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1c68daf-7f4c-4c68-ad7d-355c26646347" path="/var/lib/kubelet/pods/d1c68daf-7f4c-4c68-ad7d-355c26646347/volumes"
Jul 30 18:16:20.960748 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:20.960705    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:21.067039 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:21.066988    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:21.718775 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:21.718745    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:22.337384 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:22.337354    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:25.380734 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:25.380690    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:32.219489 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.219454    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr"]
Jul 30 18:16:32.220022 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.219671    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr" podUID="5a7af5d4-2f84-499b-9a05-e406653b25bc" containerName="maas-api" containerID="cri-o://e622baf52a72c93b67e7396a760361041c89227805bd399546a5228b1de3bfe8" gracePeriod=30
Jul 30 18:16:32.245143 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.245117    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:32.276841 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.276729    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z"]
Jul 30 18:16:32.277501 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.277233    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z" podUID="8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7" containerName="payload-processing" containerID="cri-o://a3a8aacfb81109d9931f466523dad0c28f2de8643b2b6ab3a2affadfb8524d82" gracePeriod=30
Jul 30 18:16:32.289285 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.289248    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl"]
Jul 30 18:16:32.290003 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.289972    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl" podUID="52bf434a-8de0-49bf-878f-be4e1c3651b1" containerName="payload-pre-processing" containerID="cri-o://c5788203d1509324ac7c50b0e35fdda5a194a9f10d18ff3758c16f29f9c26fb7" gracePeriod=30
Jul 30 18:16:32.331506 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.331457    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:32.396632 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.396586    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:32.460075 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.460046    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:32.583105 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.583064    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z"
Jul 30 18:16:32.591716 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.591689    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr"
Jul 30 18:16:32.634653 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.634626    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl"
Jul 30 18:16:32.744636 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.744603    2577 generic.go:356] "Generic (PLEG): container finished" podID="5a7af5d4-2f84-499b-9a05-e406653b25bc" containerID="e622baf52a72c93b67e7396a760361041c89227805bd399546a5228b1de3bfe8" exitCode=0
Jul 30 18:16:32.744846 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.744653    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr"
Jul 30 18:16:32.744846 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.744685    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr" event={"ID":"5a7af5d4-2f84-499b-9a05-e406653b25bc","Type":"ContainerDied","Data":"e622baf52a72c93b67e7396a760361041c89227805bd399546a5228b1de3bfe8"}
Jul 30 18:16:32.744846 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.744767    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr" event={"ID":"5a7af5d4-2f84-499b-9a05-e406653b25bc","Type":"ContainerDied","Data":"e435ce8b6848f44568c81c752afed3e2f5c8eff915621332adf3825648f3f720"}
Jul 30 18:16:32.745040 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.744799    2577 scope.go:117] "RemoveContainer" containerID="e622baf52a72c93b67e7396a760361041c89227805bd399546a5228b1de3bfe8"
Jul 30 18:16:32.746003 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.745983    2577 generic.go:356] "Generic (PLEG): container finished" podID="52bf434a-8de0-49bf-878f-be4e1c3651b1" containerID="c5788203d1509324ac7c50b0e35fdda5a194a9f10d18ff3758c16f29f9c26fb7" exitCode=0
Jul 30 18:16:32.746058 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.746039    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl"
Jul 30 18:16:32.746095 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.746055    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl" event={"ID":"52bf434a-8de0-49bf-878f-be4e1c3651b1","Type":"ContainerDied","Data":"c5788203d1509324ac7c50b0e35fdda5a194a9f10d18ff3758c16f29f9c26fb7"}
Jul 30 18:16:32.746144 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.746096    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl" event={"ID":"52bf434a-8de0-49bf-878f-be4e1c3651b1","Type":"ContainerDied","Data":"c9522a132643831c01443804e6dfab9a249c88f7a04e8e22597c2939f6e3a914"}
Jul 30 18:16:32.747444 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.747392    2577 generic.go:356] "Generic (PLEG): container finished" podID="8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7" containerID="a3a8aacfb81109d9931f466523dad0c28f2de8643b2b6ab3a2affadfb8524d82" exitCode=0
Jul 30 18:16:32.747444 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.747430    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z"
Jul 30 18:16:32.747444 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.747425    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z" event={"ID":"8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7","Type":"ContainerDied","Data":"a3a8aacfb81109d9931f466523dad0c28f2de8643b2b6ab3a2affadfb8524d82"}
Jul 30 18:16:32.747631 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.747473    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z" event={"ID":"8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7","Type":"ContainerDied","Data":"36d4f94260581812bce824693c9bbf9658608e5952d8c7f6b8fc9fcf4b7e41dc"}
Jul 30 18:16:32.754081 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.754063    2577 scope.go:117] "RemoveContainer" containerID="e622baf52a72c93b67e7396a760361041c89227805bd399546a5228b1de3bfe8"
Jul 30 18:16:32.754357 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:16:32.754335    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e622baf52a72c93b67e7396a760361041c89227805bd399546a5228b1de3bfe8\": container with ID starting with e622baf52a72c93b67e7396a760361041c89227805bd399546a5228b1de3bfe8 not found: ID does not exist" containerID="e622baf52a72c93b67e7396a760361041c89227805bd399546a5228b1de3bfe8"
Jul 30 18:16:32.754397 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.754365    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e622baf52a72c93b67e7396a760361041c89227805bd399546a5228b1de3bfe8"} err="failed to get container status \"e622baf52a72c93b67e7396a760361041c89227805bd399546a5228b1de3bfe8\": rpc error: code = NotFound desc = could not find container \"e622baf52a72c93b67e7396a760361041c89227805bd399546a5228b1de3bfe8\": container with ID starting with e622baf52a72c93b67e7396a760361041c89227805bd399546a5228b1de3bfe8 not found: ID does not exist"
Jul 30 18:16:32.754397 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.754382    2577 scope.go:117] "RemoveContainer" containerID="c5788203d1509324ac7c50b0e35fdda5a194a9f10d18ff3758c16f29f9c26fb7"
Jul 30 18:16:32.765321 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.765300    2577 scope.go:117] "RemoveContainer" containerID="c5788203d1509324ac7c50b0e35fdda5a194a9f10d18ff3758c16f29f9c26fb7"
Jul 30 18:16:32.765672 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:16:32.765649    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5788203d1509324ac7c50b0e35fdda5a194a9f10d18ff3758c16f29f9c26fb7\": container with ID starting with c5788203d1509324ac7c50b0e35fdda5a194a9f10d18ff3758c16f29f9c26fb7 not found: ID does not exist" containerID="c5788203d1509324ac7c50b0e35fdda5a194a9f10d18ff3758c16f29f9c26fb7"
Jul 30 18:16:32.765777 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.765678    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5788203d1509324ac7c50b0e35fdda5a194a9f10d18ff3758c16f29f9c26fb7"} err="failed to get container status \"c5788203d1509324ac7c50b0e35fdda5a194a9f10d18ff3758c16f29f9c26fb7\": rpc error: code = NotFound desc = could not find container \"c5788203d1509324ac7c50b0e35fdda5a194a9f10d18ff3758c16f29f9c26fb7\": container with ID starting with c5788203d1509324ac7c50b0e35fdda5a194a9f10d18ff3758c16f29f9c26fb7 not found: ID does not exist"
Jul 30 18:16:32.765777 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.765696    2577 scope.go:117] "RemoveContainer" containerID="a3a8aacfb81109d9931f466523dad0c28f2de8643b2b6ab3a2affadfb8524d82"
Jul 30 18:16:32.774316 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.774294    2577 scope.go:117] "RemoveContainer" containerID="a3a8aacfb81109d9931f466523dad0c28f2de8643b2b6ab3a2affadfb8524d82"
Jul 30 18:16:32.774625 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:16:32.774608    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3a8aacfb81109d9931f466523dad0c28f2de8643b2b6ab3a2affadfb8524d82\": container with ID starting with a3a8aacfb81109d9931f466523dad0c28f2de8643b2b6ab3a2affadfb8524d82 not found: ID does not exist" containerID="a3a8aacfb81109d9931f466523dad0c28f2de8643b2b6ab3a2affadfb8524d82"
Jul 30 18:16:32.774672 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.774634    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3a8aacfb81109d9931f466523dad0c28f2de8643b2b6ab3a2affadfb8524d82"} err="failed to get container status \"a3a8aacfb81109d9931f466523dad0c28f2de8643b2b6ab3a2affadfb8524d82\": rpc error: code = NotFound desc = could not find container \"a3a8aacfb81109d9931f466523dad0c28f2de8643b2b6ab3a2affadfb8524d82\": container with ID starting with a3a8aacfb81109d9931f466523dad0c28f2de8643b2b6ab3a2affadfb8524d82 not found: ID does not exist"
Jul 30 18:16:32.781028 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.781003    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7-plugins-config-volume\") pod \"8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7\" (UID: \"8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7\") "
Jul 30 18:16:32.781126 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.781057    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52bf434a-8de0-49bf-878f-be4e1c3651b1-plugins-config-volume\") pod \"52bf434a-8de0-49bf-878f-be4e1c3651b1\" (UID: \"52bf434a-8de0-49bf-878f-be4e1c3651b1\") "
Jul 30 18:16:32.781173 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.781129    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a7af5d4-2f84-499b-9a05-e406653b25bc-maas-api-tls\") pod \"5a7af5d4-2f84-499b-9a05-e406653b25bc\" (UID: \"5a7af5d4-2f84-499b-9a05-e406653b25bc\") "
Jul 30 18:16:32.781173 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.781164    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2gv7\" (UniqueName: \"kubernetes.io/projected/8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7-kube-api-access-p2gv7\") pod \"8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7\" (UID: \"8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7\") "
Jul 30 18:16:32.781260 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.781230    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dj5b\" (UniqueName: \"kubernetes.io/projected/5a7af5d4-2f84-499b-9a05-e406653b25bc-kube-api-access-8dj5b\") pod \"5a7af5d4-2f84-499b-9a05-e406653b25bc\" (UID: \"5a7af5d4-2f84-499b-9a05-e406653b25bc\") "
Jul 30 18:16:32.781260 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.781254    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9q98t\" (UniqueName: \"kubernetes.io/projected/52bf434a-8de0-49bf-878f-be4e1c3651b1-kube-api-access-9q98t\") pod \"52bf434a-8de0-49bf-878f-be4e1c3651b1\" (UID: \"52bf434a-8de0-49bf-878f-be4e1c3651b1\") "
Jul 30 18:16:32.781367 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.781293    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a7af5d4-2f84-499b-9a05-e406653b25bc-openshift-service-ca\") pod \"5a7af5d4-2f84-499b-9a05-e406653b25bc\" (UID: \"5a7af5d4-2f84-499b-9a05-e406653b25bc\") "
Jul 30 18:16:32.781424 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.781405    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7" (UID: "8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:16:32.781536 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.781510    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52bf434a-8de0-49bf-878f-be4e1c3651b1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "52bf434a-8de0-49bf-878f-be4e1c3651b1" (UID: "52bf434a-8de0-49bf-878f-be4e1c3651b1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:16:32.781696 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.781672    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:16:32.781776 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.781700    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52bf434a-8de0-49bf-878f-be4e1c3651b1-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:16:32.782075 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.781911    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a7af5d4-2f84-499b-9a05-e406653b25bc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5a7af5d4-2f84-499b-9a05-e406653b25bc" (UID: "5a7af5d4-2f84-499b-9a05-e406653b25bc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:16:32.783447 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.783419    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7-kube-api-access-p2gv7" (OuterVolumeSpecName: "kube-api-access-p2gv7") pod "8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7" (UID: "8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7"). InnerVolumeSpecName "kube-api-access-p2gv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:16:32.783548 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.783477    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a7af5d4-2f84-499b-9a05-e406653b25bc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5a7af5d4-2f84-499b-9a05-e406653b25bc" (UID: "5a7af5d4-2f84-499b-9a05-e406653b25bc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 18:16:32.783961 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.783936    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a7af5d4-2f84-499b-9a05-e406653b25bc-kube-api-access-8dj5b" (OuterVolumeSpecName: "kube-api-access-8dj5b") pod "5a7af5d4-2f84-499b-9a05-e406653b25bc" (UID: "5a7af5d4-2f84-499b-9a05-e406653b25bc"). InnerVolumeSpecName "kube-api-access-8dj5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:16:32.783961 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.783946    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52bf434a-8de0-49bf-878f-be4e1c3651b1-kube-api-access-9q98t" (OuterVolumeSpecName: "kube-api-access-9q98t") pod "52bf434a-8de0-49bf-878f-be4e1c3651b1" (UID: "52bf434a-8de0-49bf-878f-be4e1c3651b1"). InnerVolumeSpecName "kube-api-access-9q98t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:16:32.873847 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.873779    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:32.883254 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.883221    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dj5b\" (UniqueName: \"kubernetes.io/projected/5a7af5d4-2f84-499b-9a05-e406653b25bc-kube-api-access-8dj5b\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:16:32.883353 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.883256    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9q98t\" (UniqueName: \"kubernetes.io/projected/52bf434a-8de0-49bf-878f-be4e1c3651b1-kube-api-access-9q98t\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:16:32.883353 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.883268    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a7af5d4-2f84-499b-9a05-e406653b25bc-openshift-service-ca\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:16:32.883353 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.883278    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a7af5d4-2f84-499b-9a05-e406653b25bc-maas-api-tls\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:16:32.883353 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:32.883287    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2gv7\" (UniqueName: \"kubernetes.io/projected/8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7-kube-api-access-p2gv7\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:16:33.071873 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:33.071836    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl"]
Jul 30 18:16:33.075473 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:33.075444    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-984898-64b5795f96-8bpvl"]
Jul 30 18:16:33.084940 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:33.084910    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr"]
Jul 30 18:16:33.088418 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:33.088385    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-984898-f487d6d7b-ll4pr"]
Jul 30 18:16:33.097540 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:33.097504    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z"]
Jul 30 18:16:33.101431 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:33.101401    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-984898-556f7f4c8d-mts8z"]
Jul 30 18:16:34.111124 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:34.111090    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52bf434a-8de0-49bf-878f-be4e1c3651b1" path="/var/lib/kubelet/pods/52bf434a-8de0-49bf-878f-be4e1c3651b1/volumes"
Jul 30 18:16:34.111503 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:34.111481    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a7af5d4-2f84-499b-9a05-e406653b25bc" path="/var/lib/kubelet/pods/5a7af5d4-2f84-499b-9a05-e406653b25bc/volumes"
Jul 30 18:16:34.111787 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:34.111774    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7" path="/var/lib/kubelet/pods/8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7/volumes"
Jul 30 18:16:39.097763 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:39.097714    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:39.194488 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:39.194439    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:39.373609 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:39.373477    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:39.609993 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:39.609942    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:39.973109 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:39.973061    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:40.307202 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:40.307171    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:40.485229 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:40.485197    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:43.202212 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:43.202169    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:43.434065 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:43.434029    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:44.308174 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.308123    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf"]
Jul 30 18:16:44.308777 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.308760    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2de9067a-8554-48ed-888b-e99e6dca4679" containerName="payload-pre-processing"
Jul 30 18:16:44.308864 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.308781    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2de9067a-8554-48ed-888b-e99e6dca4679" containerName="payload-pre-processing"
Jul 30 18:16:44.308864 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.308793    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3" containerName="payload-processing"
Jul 30 18:16:44.308864 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.308802    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3" containerName="payload-processing"
Jul 30 18:16:44.308864 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.308850    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52bf434a-8de0-49bf-878f-be4e1c3651b1" containerName="payload-pre-processing"
Jul 30 18:16:44.308864 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.308859    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="52bf434a-8de0-49bf-878f-be4e1c3651b1" containerName="payload-pre-processing"
Jul 30 18:16:44.309035 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.308876    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1c68daf-7f4c-4c68-ad7d-355c26646347" containerName="maas-api"
Jul 30 18:16:44.309035 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.308885    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1c68daf-7f4c-4c68-ad7d-355c26646347" containerName="maas-api"
Jul 30 18:16:44.309035 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.308895    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7" containerName="payload-processing"
Jul 30 18:16:44.309035 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.308903    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7" containerName="payload-processing"
Jul 30 18:16:44.309035 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.308916    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a7af5d4-2f84-499b-9a05-e406653b25bc" containerName="maas-api"
Jul 30 18:16:44.309035 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.308924    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a7af5d4-2f84-499b-9a05-e406653b25bc" containerName="maas-api"
Jul 30 18:16:44.309035 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.309015    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2de9067a-8554-48ed-888b-e99e6dca4679" containerName="payload-pre-processing"
Jul 30 18:16:44.309035 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.309028    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="52bf434a-8de0-49bf-878f-be4e1c3651b1" containerName="payload-pre-processing"
Jul 30 18:16:44.309272 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.309039    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a7af5d4-2f84-499b-9a05-e406653b25bc" containerName="maas-api"
Jul 30 18:16:44.309272 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.309049    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1c68daf-7f4c-4c68-ad7d-355c26646347" containerName="maas-api"
Jul 30 18:16:44.309272 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.309058    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bfab8e5-3dd1-4581-a5ea-17a6edb2c2c7" containerName="payload-processing"
Jul 30 18:16:44.309272 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.309069    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4265e0c7-5b54-44d3-ac93-58bf0cf8b5c3" containerName="payload-processing"
Jul 30 18:16:44.312550 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.312530    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf"
Jul 30 18:16:44.314739 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.314714    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-d12cf5-serving-cert\""
Jul 30 18:16:44.323919 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.323896    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf"]
Jul 30 18:16:44.343733 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.343701    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:44.391974 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.391939    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8c5f5da-529b-4af1-bbd8-c56a974462ee-maas-api-tls\") pod \"maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf\" (UID: \"c8c5f5da-529b-4af1-bbd8-c56a974462ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf"
Jul 30 18:16:44.392127 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.392012    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh98b\" (UniqueName: \"kubernetes.io/projected/c8c5f5da-529b-4af1-bbd8-c56a974462ee-kube-api-access-gh98b\") pod \"maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf\" (UID: \"c8c5f5da-529b-4af1-bbd8-c56a974462ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf"
Jul 30 18:16:44.392127 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.392088    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8c5f5da-529b-4af1-bbd8-c56a974462ee-openshift-service-ca\") pod \"maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf\" (UID: \"c8c5f5da-529b-4af1-bbd8-c56a974462ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf"
Jul 30 18:16:44.493244 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.493207    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh98b\" (UniqueName: \"kubernetes.io/projected/c8c5f5da-529b-4af1-bbd8-c56a974462ee-kube-api-access-gh98b\") pod \"maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf\" (UID: \"c8c5f5da-529b-4af1-bbd8-c56a974462ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf"
Jul 30 18:16:44.493428 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.493285    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8c5f5da-529b-4af1-bbd8-c56a974462ee-openshift-service-ca\") pod \"maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf\" (UID: \"c8c5f5da-529b-4af1-bbd8-c56a974462ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf"
Jul 30 18:16:44.493428 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.493311    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8c5f5da-529b-4af1-bbd8-c56a974462ee-maas-api-tls\") pod \"maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf\" (UID: \"c8c5f5da-529b-4af1-bbd8-c56a974462ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf"
Jul 30 18:16:44.493428 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:16:44.493411    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-serving-cert: secret "maas-api-e2e-api-a-d12cf5-serving-cert" not found
Jul 30 18:16:44.493539 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:16:44.493470    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8c5f5da-529b-4af1-bbd8-c56a974462ee-maas-api-tls podName:c8c5f5da-529b-4af1-bbd8-c56a974462ee nodeName:}" failed. No retries permitted until 2026-07-30 18:16:44.993453342 +0000 UTC m=+2979.447017310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c8c5f5da-529b-4af1-bbd8-c56a974462ee-maas-api-tls") pod "maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf" (UID: "c8c5f5da-529b-4af1-bbd8-c56a974462ee") : secret "maas-api-e2e-api-a-d12cf5-serving-cert" not found
Jul 30 18:16:44.493963 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.493944    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8c5f5da-529b-4af1-bbd8-c56a974462ee-openshift-service-ca\") pod \"maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf\" (UID: \"c8c5f5da-529b-4af1-bbd8-c56a974462ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf"
Jul 30 18:16:44.502365 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.502339    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh98b\" (UniqueName: \"kubernetes.io/projected/c8c5f5da-529b-4af1-bbd8-c56a974462ee-kube-api-access-gh98b\") pod \"maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf\" (UID: \"c8c5f5da-529b-4af1-bbd8-c56a974462ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf"
Jul 30 18:16:44.852775 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.852741    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:44.947942 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.947911    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:44.966423 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.966392    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9"]
Jul 30 18:16:44.969989 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.969972    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9"
Jul 30 18:16:44.972458 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.972435    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-d12cf5\""
Jul 30 18:16:44.972579 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.972499    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-d12cf5-dockercfg-gwb4k\""
Jul 30 18:16:44.979721 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.979696    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9"]
Jul 30 18:16:44.989336 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.989312    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m"]
Jul 30 18:16:44.993043 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.993026    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m"
Jul 30 18:16:44.997138 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:44.997118    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8c5f5da-529b-4af1-bbd8-c56a974462ee-maas-api-tls\") pod \"maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf\" (UID: \"c8c5f5da-529b-4af1-bbd8-c56a974462ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf"
Jul 30 18:16:44.997290 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:16:44.997272    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-serving-cert: secret "maas-api-e2e-api-a-d12cf5-serving-cert" not found
Jul 30 18:16:44.997351 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:16:44.997340    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8c5f5da-529b-4af1-bbd8-c56a974462ee-maas-api-tls podName:c8c5f5da-529b-4af1-bbd8-c56a974462ee nodeName:}" failed. No retries permitted until 2026-07-30 18:16:45.99732182 +0000 UTC m=+2980.450885811 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c8c5f5da-529b-4af1-bbd8-c56a974462ee-maas-api-tls") pod "maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf" (UID: "c8c5f5da-529b-4af1-bbd8-c56a974462ee") : secret "maas-api-e2e-api-a-d12cf5-serving-cert" not found
Jul 30 18:16:45.004411 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.004373    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m"]
Jul 30 18:16:45.098024 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.097989    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e9d3ec1-7ee1-4615-820c-415c02ce5600-plugins-config-volume\") pod \"payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m\" (UID: \"3e9d3ec1-7ee1-4615-820c-415c02ce5600\") " pod="openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m"
Jul 30 18:16:45.098234 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.098035    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm7wn\" (UniqueName: \"kubernetes.io/projected/3e9d3ec1-7ee1-4615-820c-415c02ce5600-kube-api-access-sm7wn\") pod \"payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m\" (UID: \"3e9d3ec1-7ee1-4615-820c-415c02ce5600\") " pod="openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m"
Jul 30 18:16:45.098234 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.098074    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlh9v\" (UniqueName: \"kubernetes.io/projected/496f7c93-8b7c-429d-8a63-9ce26e160ddf-kube-api-access-hlh9v\") pod \"payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9\" (UID: \"496f7c93-8b7c-429d-8a63-9ce26e160ddf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9"
Jul 30 18:16:45.098234 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.098152    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/496f7c93-8b7c-429d-8a63-9ce26e160ddf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9\" (UID: \"496f7c93-8b7c-429d-8a63-9ce26e160ddf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9"
Jul 30 18:16:45.199397 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.199300    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlh9v\" (UniqueName: \"kubernetes.io/projected/496f7c93-8b7c-429d-8a63-9ce26e160ddf-kube-api-access-hlh9v\") pod \"payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9\" (UID: \"496f7c93-8b7c-429d-8a63-9ce26e160ddf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9"
Jul 30 18:16:45.199397 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.199354    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/496f7c93-8b7c-429d-8a63-9ce26e160ddf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9\" (UID: \"496f7c93-8b7c-429d-8a63-9ce26e160ddf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9"
Jul 30 18:16:45.199654 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.199447    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e9d3ec1-7ee1-4615-820c-415c02ce5600-plugins-config-volume\") pod \"payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m\" (UID: \"3e9d3ec1-7ee1-4615-820c-415c02ce5600\") " pod="openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m"
Jul 30 18:16:45.199654 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.199486    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm7wn\" (UniqueName: \"kubernetes.io/projected/3e9d3ec1-7ee1-4615-820c-415c02ce5600-kube-api-access-sm7wn\") pod \"payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m\" (UID: \"3e9d3ec1-7ee1-4615-820c-415c02ce5600\") " pod="openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m"
Jul 30 18:16:45.200216 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.200197    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/496f7c93-8b7c-429d-8a63-9ce26e160ddf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9\" (UID: \"496f7c93-8b7c-429d-8a63-9ce26e160ddf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9"
Jul 30 18:16:45.200282 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.200197    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e9d3ec1-7ee1-4615-820c-415c02ce5600-plugins-config-volume\") pod \"payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m\" (UID: \"3e9d3ec1-7ee1-4615-820c-415c02ce5600\") " pod="openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m"
Jul 30 18:16:45.207780 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.207745    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm7wn\" (UniqueName: \"kubernetes.io/projected/3e9d3ec1-7ee1-4615-820c-415c02ce5600-kube-api-access-sm7wn\") pod \"payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m\" (UID: \"3e9d3ec1-7ee1-4615-820c-415c02ce5600\") " pod="openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m"
Jul 30 18:16:45.208088 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.208066    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlh9v\" (UniqueName: \"kubernetes.io/projected/496f7c93-8b7c-429d-8a63-9ce26e160ddf-kube-api-access-hlh9v\") pod \"payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9\" (UID: \"496f7c93-8b7c-429d-8a63-9ce26e160ddf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9"
Jul 30 18:16:45.283065 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.283024    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9"
Jul 30 18:16:45.306075 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.305971    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m"
Jul 30 18:16:45.435976 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.435913    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9"]
Jul 30 18:16:45.437194 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:16:45.437169    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod496f7c93_8b7c_429d_8a63_9ce26e160ddf.slice/crio-34026b58a34ded69840e79b97438ae5385a4c77916d4b83a12dfe94cf6aa3c70 WatchSource:0}: Error finding container 34026b58a34ded69840e79b97438ae5385a4c77916d4b83a12dfe94cf6aa3c70: Status 404 returned error can't find the container with id 34026b58a34ded69840e79b97438ae5385a4c77916d4b83a12dfe94cf6aa3c70
Jul 30 18:16:45.468463 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.468435    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m"]
Jul 30 18:16:45.474986 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:16:45.474957    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e9d3ec1_7ee1_4615_820c_415c02ce5600.slice/crio-d53c4146f4fb59ca5993e13bd55b471df45078c302123eb6f6b55f6d319c066d WatchSource:0}: Error finding container d53c4146f4fb59ca5993e13bd55b471df45078c302123eb6f6b55f6d319c066d: Status 404 returned error can't find the container with id d53c4146f4fb59ca5993e13bd55b471df45078c302123eb6f6b55f6d319c066d
Jul 30 18:16:45.519312 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.519282    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:16:45.832496 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.832457    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m" event={"ID":"3e9d3ec1-7ee1-4615-820c-415c02ce5600","Type":"ContainerStarted","Data":"9128144bbb871ae107ed98f33892864c361d8e0938a8fa2d454ce4ad7da484bb"}
Jul 30 18:16:45.832689 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.832519    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m" event={"ID":"3e9d3ec1-7ee1-4615-820c-415c02ce5600","Type":"ContainerStarted","Data":"d53c4146f4fb59ca5993e13bd55b471df45078c302123eb6f6b55f6d319c066d"}
Jul 30 18:16:45.832689 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.832649    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m"
Jul 30 18:16:45.833999 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.833968    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9" event={"ID":"496f7c93-8b7c-429d-8a63-9ce26e160ddf","Type":"ContainerStarted","Data":"be298e0d790d78fc64b4c95c2ad4bcb6bf5e1a76f494677e0a493270806dfdb0"}
Jul 30 18:16:45.834119 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.834006    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9" event={"ID":"496f7c93-8b7c-429d-8a63-9ce26e160ddf","Type":"ContainerStarted","Data":"34026b58a34ded69840e79b97438ae5385a4c77916d4b83a12dfe94cf6aa3c70"}
Jul 30 18:16:45.834182 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.834147    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9"
Jul 30 18:16:45.848237 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.848194    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m" podStartSLOduration=1.848180459 podStartE2EDuration="1.848180459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:16:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:16:45.848042564 +0000 UTC m=+2980.301606554" watchObservedRunningTime="2026-07-30 18:16:45.848180459 +0000 UTC m=+2980.301744449"
Jul 30 18:16:45.863779 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:45.863721    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9" podStartSLOduration=1.863700401 podStartE2EDuration="1.863700401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:16:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:16:45.862458505 +0000 UTC m=+2980.316022506" watchObservedRunningTime="2026-07-30 18:16:45.863700401 +0000 UTC m=+2980.317264396"
Jul 30 18:16:46.008736 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:46.008703    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8c5f5da-529b-4af1-bbd8-c56a974462ee-maas-api-tls\") pod \"maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf\" (UID: \"c8c5f5da-529b-4af1-bbd8-c56a974462ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf"
Jul 30 18:16:46.011247 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:46.011228    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8c5f5da-529b-4af1-bbd8-c56a974462ee-maas-api-tls\") pod \"maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf\" (UID: \"c8c5f5da-529b-4af1-bbd8-c56a974462ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf"
Jul 30 18:16:46.126119 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:46.126037    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf"
Jul 30 18:16:46.263832 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:46.263787    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf"]
Jul 30 18:16:46.266024 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:16:46.265970    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8c5f5da_529b_4af1_bbd8_c56a974462ee.slice/crio-8c2bf6976877d9c56740c5ba0ec7d2a1fcfcbd9cdf229cffc9b376f17bb0335e WatchSource:0}: Error finding container 8c2bf6976877d9c56740c5ba0ec7d2a1fcfcbd9cdf229cffc9b376f17bb0335e: Status 404 returned error can't find the container with id 8c2bf6976877d9c56740c5ba0ec7d2a1fcfcbd9cdf229cffc9b376f17bb0335e
Jul 30 18:16:46.267609 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:46.267589    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 18:16:46.850236 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:46.850206    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf" event={"ID":"c8c5f5da-529b-4af1-bbd8-c56a974462ee","Type":"ContainerStarted","Data":"8c2bf6976877d9c56740c5ba0ec7d2a1fcfcbd9cdf229cffc9b376f17bb0335e"}
Jul 30 18:16:47.857210 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:47.857175    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf" event={"ID":"c8c5f5da-529b-4af1-bbd8-c56a974462ee","Type":"ContainerStarted","Data":"4a567ff58a4f3ba61b3bd87c536176f3f0e8f889881e5848035b30bad54fe901"}
Jul 30 18:16:47.857663 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:47.857324    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf"
Jul 30 18:16:47.877159 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:47.877107    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf" podStartSLOduration=3.323212205 podStartE2EDuration="3.877089711s" totalImagesPullingTime="553.877506ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:16:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:16:47.874134565 +0000 UTC m=+2982.327698567" watchObservedRunningTime="2026-07-30 18:16:47.877089711 +0000 UTC m=+2982.330653702"
Jul 30 18:16:53.868712 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:53.868684    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf"
Jul 30 18:16:56.852785 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:56.852755    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9"
Jul 30 18:16:56.853352 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:16:56.853062    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m"
Jul 30 18:17:08.273117 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:08.273078    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:17:08.780717 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:08.780684    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:17:09.143326 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:09.143240    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:17:09.243024 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:09.242989    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:17:12.341715 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:12.341684    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:17:12.436547 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:12.436517    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:17:13.505575 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:13.505499    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs"]
Jul 30 18:17:13.510958 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:13.510933    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs"
Jul 30 18:17:13.512928 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:13.512905    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-b51ede-serving-cert\""
Jul 30 18:17:13.518806 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:13.518786    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs"]
Jul 30 18:17:13.544882 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:13.544857    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:17:13.588286 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:13.588244    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/947fe0f8-2b55-4a63-949a-4f20aa646f57-maas-api-tls\") pod \"maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs\" (UID: \"947fe0f8-2b55-4a63-949a-4f20aa646f57\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs"
Jul 30 18:17:13.588286 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:13.588287    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv2bc\" (UniqueName: \"kubernetes.io/projected/947fe0f8-2b55-4a63-949a-4f20aa646f57-kube-api-access-zv2bc\") pod \"maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs\" (UID: \"947fe0f8-2b55-4a63-949a-4f20aa646f57\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs"
Jul 30 18:17:13.588534 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:13.588449    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/947fe0f8-2b55-4a63-949a-4f20aa646f57-openshift-service-ca\") pod \"maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs\" (UID: \"947fe0f8-2b55-4a63-949a-4f20aa646f57\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs"
Jul 30 18:17:13.684175 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:13.684143    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:17:13.689574 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:13.689553    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/947fe0f8-2b55-4a63-949a-4f20aa646f57-openshift-service-ca\") pod \"maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs\" (UID: \"947fe0f8-2b55-4a63-949a-4f20aa646f57\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs"
Jul 30 18:17:13.689655 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:13.689610    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/947fe0f8-2b55-4a63-949a-4f20aa646f57-maas-api-tls\") pod \"maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs\" (UID: \"947fe0f8-2b55-4a63-949a-4f20aa646f57\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs"
Jul 30 18:17:13.689655 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:13.689630    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv2bc\" (UniqueName: \"kubernetes.io/projected/947fe0f8-2b55-4a63-949a-4f20aa646f57-kube-api-access-zv2bc\") pod \"maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs\" (UID: \"947fe0f8-2b55-4a63-949a-4f20aa646f57\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs"
Jul 30 18:17:13.690244 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:13.690224    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/947fe0f8-2b55-4a63-949a-4f20aa646f57-openshift-service-ca\") pod \"maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs\" (UID: \"947fe0f8-2b55-4a63-949a-4f20aa646f57\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs"
Jul 30 18:17:13.692162 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:13.692141    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/947fe0f8-2b55-4a63-949a-4f20aa646f57-maas-api-tls\") pod \"maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs\" (UID: \"947fe0f8-2b55-4a63-949a-4f20aa646f57\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs"
Jul 30 18:17:13.697959 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:13.697937    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv2bc\" (UniqueName: \"kubernetes.io/projected/947fe0f8-2b55-4a63-949a-4f20aa646f57-kube-api-access-zv2bc\") pod \"maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs\" (UID: \"947fe0f8-2b55-4a63-949a-4f20aa646f57\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs"
Jul 30 18:17:13.824716 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:13.824622    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs"
Jul 30 18:17:13.970398 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:13.970242    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs"]
Jul 30 18:17:14.002759 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:14.002718    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs" event={"ID":"947fe0f8-2b55-4a63-949a-4f20aa646f57","Type":"ContainerStarted","Data":"fcdd39f02a7b44482eba9fc566be53ff3ee81eead44d9df064fc3c3609ebf041"}
Jul 30 18:17:14.026559 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:14.026519    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:17:14.121768 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:14.121646    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:17:14.128216 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:14.128189    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz"]
Jul 30 18:17:14.133072 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:14.133056    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz"
Jul 30 18:17:14.135937 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:14.135914    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-b51ede-dockercfg-mbmzw\""
Jul 30 18:17:14.136062 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:14.135933    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-b51ede\""
Jul 30 18:17:14.144619 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:14.144596    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz"]
Jul 30 18:17:14.195654 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:14.195612    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d034ebda-561e-4dcb-bd56-60918690d7e6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz\" (UID: \"d034ebda-561e-4dcb-bd56-60918690d7e6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz"
Jul 30 18:17:14.195840 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:14.195668    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvn48\" (UniqueName: \"kubernetes.io/projected/d034ebda-561e-4dcb-bd56-60918690d7e6-kube-api-access-pvn48\") pod \"payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz\" (UID: \"d034ebda-561e-4dcb-bd56-60918690d7e6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz"
Jul 30 18:17:14.297168 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:14.297126    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d034ebda-561e-4dcb-bd56-60918690d7e6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz\" (UID: \"d034ebda-561e-4dcb-bd56-60918690d7e6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz"
Jul 30 18:17:14.297341 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:14.297188    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvn48\" (UniqueName: \"kubernetes.io/projected/d034ebda-561e-4dcb-bd56-60918690d7e6-kube-api-access-pvn48\") pod \"payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz\" (UID: \"d034ebda-561e-4dcb-bd56-60918690d7e6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz"
Jul 30 18:17:14.297861 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:14.297840    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d034ebda-561e-4dcb-bd56-60918690d7e6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz\" (UID: \"d034ebda-561e-4dcb-bd56-60918690d7e6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz"
Jul 30 18:17:14.306014 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:14.305988    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvn48\" (UniqueName: \"kubernetes.io/projected/d034ebda-561e-4dcb-bd56-60918690d7e6-kube-api-access-pvn48\") pod \"payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz\" (UID: \"d034ebda-561e-4dcb-bd56-60918690d7e6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz"
Jul 30 18:17:14.445077 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:14.445050    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz"
Jul 30 18:17:14.790887 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:14.790862    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz"]
Jul 30 18:17:14.793402 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:17:14.793372    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd034ebda_561e_4dcb_bd56_60918690d7e6.slice/crio-7583a14b3a38aa7bfb4d17a03eddcfd32594e8d0d06f32325ae4a725730241bc WatchSource:0}: Error finding container 7583a14b3a38aa7bfb4d17a03eddcfd32594e8d0d06f32325ae4a725730241bc: Status 404 returned error can't find the container with id 7583a14b3a38aa7bfb4d17a03eddcfd32594e8d0d06f32325ae4a725730241bc
Jul 30 18:17:15.010655 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:15.010546    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz" event={"ID":"d034ebda-561e-4dcb-bd56-60918690d7e6","Type":"ContainerStarted","Data":"2cc5e3921724b37a4ddf3c5a4685d9a04a208233ea7af9cba2e4b1b84d819b42"}
Jul 30 18:17:15.010655 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:15.010615    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz" event={"ID":"d034ebda-561e-4dcb-bd56-60918690d7e6","Type":"ContainerStarted","Data":"7583a14b3a38aa7bfb4d17a03eddcfd32594e8d0d06f32325ae4a725730241bc"}
Jul 30 18:17:15.010945 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:15.010806    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz"
Jul 30 18:17:15.012452 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:15.012426    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs" event={"ID":"947fe0f8-2b55-4a63-949a-4f20aa646f57","Type":"ContainerStarted","Data":"8e3271d076c9cf09d4f59b9ff162161ffd765eb02222d2ef7b1c7bac429f6d1c"}
Jul 30 18:17:15.012702 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:15.012510    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs"
Jul 30 18:17:15.027202 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:15.027156    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz" podStartSLOduration=1.027141563 podStartE2EDuration="1.027141563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:17:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:17:15.025497966 +0000 UTC m=+3009.479061958" watchObservedRunningTime="2026-07-30 18:17:15.027141563 +0000 UTC m=+3009.480705582"
Jul 30 18:17:15.041160 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:15.041073    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs" podStartSLOduration=1.65895313 podStartE2EDuration="2.041059163s" totalImagesPullingTime="382.106033ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:17:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:17:15.039216606 +0000 UTC m=+3009.492780599" watchObservedRunningTime="2026-07-30 18:17:15.041059163 +0000 UTC m=+3009.494623153"
Jul 30 18:17:21.023702 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:21.023672    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs"
Jul 30 18:17:26.019953 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:26.019906    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz" podUID="d034ebda-561e-4dcb-bd56-60918690d7e6" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.79:9004: connect: connection refused"
Jul 30 18:17:34.445646 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:34.445597    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz" podUID="d034ebda-561e-4dcb-bd56-60918690d7e6" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.79:9004: connect: connection refused"
Jul 30 18:17:36.019783 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:36.019754    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz"
Jul 30 18:17:37.373941 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:37.373905    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:17:55.849984 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:55.849952    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf"]
Jul 30 18:17:55.850576 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:55.850196    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf" podUID="c8c5f5da-529b-4af1-bbd8-c56a974462ee" containerName="maas-api" containerID="cri-o://4a567ff58a4f3ba61b3bd87c536176f3f0e8f889881e5848035b30bad54fe901" gracePeriod=30
Jul 30 18:17:55.880916 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:55.880889    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:17:55.913403 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:55.913360    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m"]
Jul 30 18:17:55.913671 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:55.913639    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m" podUID="3e9d3ec1-7ee1-4615-820c-415c02ce5600" containerName="payload-processing" containerID="cri-o://9128144bbb871ae107ed98f33892864c361d8e0938a8fa2d454ce4ad7da484bb" gracePeriod=30
Jul 30 18:17:55.921014 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:55.920985    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9"]
Jul 30 18:17:55.921338 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:55.921227    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9" podUID="496f7c93-8b7c-429d-8a63-9ce26e160ddf" containerName="payload-pre-processing" containerID="cri-o://be298e0d790d78fc64b4c95c2ad4bcb6bf5e1a76f494677e0a493270806dfdb0" gracePeriod=30
Jul 30 18:17:55.961219 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:55.961134    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:17:56.026935 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.026886    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:17:56.091852 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.091772    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:17:56.239359 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.239070    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m"
Jul 30 18:17:56.254995 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.254602    2577 generic.go:356] "Generic (PLEG): container finished" podID="c8c5f5da-529b-4af1-bbd8-c56a974462ee" containerID="4a567ff58a4f3ba61b3bd87c536176f3f0e8f889881e5848035b30bad54fe901" exitCode=0
Jul 30 18:17:56.254995 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.254743    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf" event={"ID":"c8c5f5da-529b-4af1-bbd8-c56a974462ee","Type":"ContainerDied","Data":"4a567ff58a4f3ba61b3bd87c536176f3f0e8f889881e5848035b30bad54fe901"}
Jul 30 18:17:56.254995 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.254791    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf" event={"ID":"c8c5f5da-529b-4af1-bbd8-c56a974462ee","Type":"ContainerDied","Data":"8c2bf6976877d9c56740c5ba0ec7d2a1fcfcbd9cdf229cffc9b376f17bb0335e"}
Jul 30 18:17:56.267902 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.267534    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c2bf6976877d9c56740c5ba0ec7d2a1fcfcbd9cdf229cffc9b376f17bb0335e"
Jul 30 18:17:56.270517 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.270027    2577 generic.go:356] "Generic (PLEG): container finished" podID="3e9d3ec1-7ee1-4615-820c-415c02ce5600" containerID="9128144bbb871ae107ed98f33892864c361d8e0938a8fa2d454ce4ad7da484bb" exitCode=0
Jul 30 18:17:56.270517 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.270072    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m"
Jul 30 18:17:56.270517 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.270108    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m" event={"ID":"3e9d3ec1-7ee1-4615-820c-415c02ce5600","Type":"ContainerDied","Data":"9128144bbb871ae107ed98f33892864c361d8e0938a8fa2d454ce4ad7da484bb"}
Jul 30 18:17:56.270517 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.270153    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m" event={"ID":"3e9d3ec1-7ee1-4615-820c-415c02ce5600","Type":"ContainerDied","Data":"d53c4146f4fb59ca5993e13bd55b471df45078c302123eb6f6b55f6d319c066d"}
Jul 30 18:17:56.270517 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.270183    2577 scope.go:117] "RemoveContainer" containerID="9128144bbb871ae107ed98f33892864c361d8e0938a8fa2d454ce4ad7da484bb"
Jul 30 18:17:56.272135 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.272118    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf"
Jul 30 18:17:56.272243 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.272211    2577 generic.go:356] "Generic (PLEG): container finished" podID="496f7c93-8b7c-429d-8a63-9ce26e160ddf" containerID="be298e0d790d78fc64b4c95c2ad4bcb6bf5e1a76f494677e0a493270806dfdb0" exitCode=0
Jul 30 18:17:56.272313 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.272252    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9" event={"ID":"496f7c93-8b7c-429d-8a63-9ce26e160ddf","Type":"ContainerDied","Data":"be298e0d790d78fc64b4c95c2ad4bcb6bf5e1a76f494677e0a493270806dfdb0"}
Jul 30 18:17:56.279056 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.279020    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9"
Jul 30 18:17:56.280540 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.280521    2577 scope.go:117] "RemoveContainer" containerID="9128144bbb871ae107ed98f33892864c361d8e0938a8fa2d454ce4ad7da484bb"
Jul 30 18:17:56.280828 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:17:56.280786    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9128144bbb871ae107ed98f33892864c361d8e0938a8fa2d454ce4ad7da484bb\": container with ID starting with 9128144bbb871ae107ed98f33892864c361d8e0938a8fa2d454ce4ad7da484bb not found: ID does not exist" containerID="9128144bbb871ae107ed98f33892864c361d8e0938a8fa2d454ce4ad7da484bb"
Jul 30 18:17:56.280931 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.280829    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9128144bbb871ae107ed98f33892864c361d8e0938a8fa2d454ce4ad7da484bb"} err="failed to get container status \"9128144bbb871ae107ed98f33892864c361d8e0938a8fa2d454ce4ad7da484bb\": rpc error: code = NotFound desc = could not find container \"9128144bbb871ae107ed98f33892864c361d8e0938a8fa2d454ce4ad7da484bb\": container with ID starting with 9128144bbb871ae107ed98f33892864c361d8e0938a8fa2d454ce4ad7da484bb not found: ID does not exist"
Jul 30 18:17:56.308733 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.308707    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8c5f5da-529b-4af1-bbd8-c56a974462ee-openshift-service-ca\") pod \"c8c5f5da-529b-4af1-bbd8-c56a974462ee\" (UID: \"c8c5f5da-529b-4af1-bbd8-c56a974462ee\") "
Jul 30 18:17:56.308930 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.308745    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sm7wn\" (UniqueName: \"kubernetes.io/projected/3e9d3ec1-7ee1-4615-820c-415c02ce5600-kube-api-access-sm7wn\") pod \"3e9d3ec1-7ee1-4615-820c-415c02ce5600\" (UID: \"3e9d3ec1-7ee1-4615-820c-415c02ce5600\") "
Jul 30 18:17:56.308930 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.308778    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gh98b\" (UniqueName: \"kubernetes.io/projected/c8c5f5da-529b-4af1-bbd8-c56a974462ee-kube-api-access-gh98b\") pod \"c8c5f5da-529b-4af1-bbd8-c56a974462ee\" (UID: \"c8c5f5da-529b-4af1-bbd8-c56a974462ee\") "
Jul 30 18:17:56.308930 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.308806    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8c5f5da-529b-4af1-bbd8-c56a974462ee-maas-api-tls\") pod \"c8c5f5da-529b-4af1-bbd8-c56a974462ee\" (UID: \"c8c5f5da-529b-4af1-bbd8-c56a974462ee\") "
Jul 30 18:17:56.309120 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.308941    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/496f7c93-8b7c-429d-8a63-9ce26e160ddf-plugins-config-volume\") pod \"496f7c93-8b7c-429d-8a63-9ce26e160ddf\" (UID: \"496f7c93-8b7c-429d-8a63-9ce26e160ddf\") "
Jul 30 18:17:56.309120 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.308973    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e9d3ec1-7ee1-4615-820c-415c02ce5600-plugins-config-volume\") pod \"3e9d3ec1-7ee1-4615-820c-415c02ce5600\" (UID: \"3e9d3ec1-7ee1-4615-820c-415c02ce5600\") "
Jul 30 18:17:56.309120 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.309031    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlh9v\" (UniqueName: \"kubernetes.io/projected/496f7c93-8b7c-429d-8a63-9ce26e160ddf-kube-api-access-hlh9v\") pod \"496f7c93-8b7c-429d-8a63-9ce26e160ddf\" (UID: \"496f7c93-8b7c-429d-8a63-9ce26e160ddf\") "
Jul 30 18:17:56.310394 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.309652    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/496f7c93-8b7c-429d-8a63-9ce26e160ddf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "496f7c93-8b7c-429d-8a63-9ce26e160ddf" (UID: "496f7c93-8b7c-429d-8a63-9ce26e160ddf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:17:56.310394 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.309997    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e9d3ec1-7ee1-4615-820c-415c02ce5600-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3e9d3ec1-7ee1-4615-820c-415c02ce5600" (UID: "3e9d3ec1-7ee1-4615-820c-415c02ce5600"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:17:56.310592 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.310549    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8c5f5da-529b-4af1-bbd8-c56a974462ee-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c8c5f5da-529b-4af1-bbd8-c56a974462ee" (UID: "c8c5f5da-529b-4af1-bbd8-c56a974462ee"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:17:56.312466 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.312424    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8c5f5da-529b-4af1-bbd8-c56a974462ee-kube-api-access-gh98b" (OuterVolumeSpecName: "kube-api-access-gh98b") pod "c8c5f5da-529b-4af1-bbd8-c56a974462ee" (UID: "c8c5f5da-529b-4af1-bbd8-c56a974462ee"). InnerVolumeSpecName "kube-api-access-gh98b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:17:56.312591 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.312552    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8c5f5da-529b-4af1-bbd8-c56a974462ee-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c8c5f5da-529b-4af1-bbd8-c56a974462ee" (UID: "c8c5f5da-529b-4af1-bbd8-c56a974462ee"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 18:17:56.312899 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.312870    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/496f7c93-8b7c-429d-8a63-9ce26e160ddf-kube-api-access-hlh9v" (OuterVolumeSpecName: "kube-api-access-hlh9v") pod "496f7c93-8b7c-429d-8a63-9ce26e160ddf" (UID: "496f7c93-8b7c-429d-8a63-9ce26e160ddf"). InnerVolumeSpecName "kube-api-access-hlh9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:17:56.313274 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.313249    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e9d3ec1-7ee1-4615-820c-415c02ce5600-kube-api-access-sm7wn" (OuterVolumeSpecName: "kube-api-access-sm7wn") pod "3e9d3ec1-7ee1-4615-820c-415c02ce5600" (UID: "3e9d3ec1-7ee1-4615-820c-415c02ce5600"). InnerVolumeSpecName "kube-api-access-sm7wn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:17:56.410245 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.410154    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlh9v\" (UniqueName: \"kubernetes.io/projected/496f7c93-8b7c-429d-8a63-9ce26e160ddf-kube-api-access-hlh9v\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:17:56.410245 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.410186    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8c5f5da-529b-4af1-bbd8-c56a974462ee-openshift-service-ca\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:17:56.410245 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.410197    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sm7wn\" (UniqueName: \"kubernetes.io/projected/3e9d3ec1-7ee1-4615-820c-415c02ce5600-kube-api-access-sm7wn\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:17:56.410245 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.410207    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gh98b\" (UniqueName: \"kubernetes.io/projected/c8c5f5da-529b-4af1-bbd8-c56a974462ee-kube-api-access-gh98b\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:17:56.410245 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.410219    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8c5f5da-529b-4af1-bbd8-c56a974462ee-maas-api-tls\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:17:56.410245 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.410229    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/496f7c93-8b7c-429d-8a63-9ce26e160ddf-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:17:56.410245 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.410237    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e9d3ec1-7ee1-4615-820c-415c02ce5600-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:17:56.484409 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.484373    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:17:56.594739 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.594705    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m"]
Jul 30 18:17:56.599767 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:56.599742    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d12cf5-5897dc496b-4bt7m"]
Jul 30 18:17:57.278645 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:57.278620    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9"
Jul 30 18:17:57.279120 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:57.278644    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9" event={"ID":"496f7c93-8b7c-429d-8a63-9ce26e160ddf","Type":"ContainerDied","Data":"34026b58a34ded69840e79b97438ae5385a4c77916d4b83a12dfe94cf6aa3c70"}
Jul 30 18:17:57.279120 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:57.278705    2577 scope.go:117] "RemoveContainer" containerID="be298e0d790d78fc64b4c95c2ad4bcb6bf5e1a76f494677e0a493270806dfdb0"
Jul 30 18:17:57.280045 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:57.279912    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf"
Jul 30 18:17:57.304239 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:57.304214    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf"]
Jul 30 18:17:57.309969 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:57.309929    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d12cf5-6dff77ddb7-c5jnf"]
Jul 30 18:17:57.323683 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:57.323653    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9"]
Jul 30 18:17:57.328059 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:57.328034    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d12cf5-8848d45c6-m8mq9"]
Jul 30 18:17:58.111260 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:58.111230    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e9d3ec1-7ee1-4615-820c-415c02ce5600" path="/var/lib/kubelet/pods/3e9d3ec1-7ee1-4615-820c-415c02ce5600/volumes"
Jul 30 18:17:58.111609 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:58.111596    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="496f7c93-8b7c-429d-8a63-9ce26e160ddf" path="/var/lib/kubelet/pods/496f7c93-8b7c-429d-8a63-9ce26e160ddf/volumes"
Jul 30 18:17:58.111962 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:17:58.111950    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8c5f5da-529b-4af1-bbd8-c56a974462ee" path="/var/lib/kubelet/pods/c8c5f5da-529b-4af1-bbd8-c56a974462ee/volumes"
Jul 30 18:18:02.448779 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:02.448746    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:02.570662 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:02.570625    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:02.734360 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:02.734260    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:02.998487 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:02.998452    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:06.993998 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:06.993963    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:07.046365 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:07.046322    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:18.698785 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:18.698746    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs"]
Jul 30 18:18:18.699288 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:18.699016    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs" podUID="947fe0f8-2b55-4a63-949a-4f20aa646f57" containerName="maas-api" containerID="cri-o://8e3271d076c9cf09d4f59b9ff162161ffd765eb02222d2ef7b1c7bac429f6d1c" gracePeriod=30
Jul 30 18:18:18.733793 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:18.733759    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:18.770323 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:18.770293    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz"]
Jul 30 18:18:18.770575 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:18.770549    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz" podUID="d034ebda-561e-4dcb-bd56-60918690d7e6" containerName="payload-pre-processing" containerID="cri-o://2cc5e3921724b37a4ddf3c5a4685d9a04a208233ea7af9cba2e4b1b84d819b42" gracePeriod=30
Jul 30 18:18:18.829984 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:18.829955    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:18.924251 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:18.924211    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:18.987620 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:18.987585    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:19.021539 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.021512    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs"
Jul 30 18:18:19.050526 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.050503    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz"
Jul 30 18:18:19.127631 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.127596    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/947fe0f8-2b55-4a63-949a-4f20aa646f57-openshift-service-ca\") pod \"947fe0f8-2b55-4a63-949a-4f20aa646f57\" (UID: \"947fe0f8-2b55-4a63-949a-4f20aa646f57\") "
Jul 30 18:18:19.127878 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.127650    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/947fe0f8-2b55-4a63-949a-4f20aa646f57-maas-api-tls\") pod \"947fe0f8-2b55-4a63-949a-4f20aa646f57\" (UID: \"947fe0f8-2b55-4a63-949a-4f20aa646f57\") "
Jul 30 18:18:19.127878 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.127771    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zv2bc\" (UniqueName: \"kubernetes.io/projected/947fe0f8-2b55-4a63-949a-4f20aa646f57-kube-api-access-zv2bc\") pod \"947fe0f8-2b55-4a63-949a-4f20aa646f57\" (UID: \"947fe0f8-2b55-4a63-949a-4f20aa646f57\") "
Jul 30 18:18:19.128083 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.128060    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/947fe0f8-2b55-4a63-949a-4f20aa646f57-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "947fe0f8-2b55-4a63-949a-4f20aa646f57" (UID: "947fe0f8-2b55-4a63-949a-4f20aa646f57"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:18:19.129976 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.129956    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/947fe0f8-2b55-4a63-949a-4f20aa646f57-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "947fe0f8-2b55-4a63-949a-4f20aa646f57" (UID: "947fe0f8-2b55-4a63-949a-4f20aa646f57"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 18:18:19.130049 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.130007    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/947fe0f8-2b55-4a63-949a-4f20aa646f57-kube-api-access-zv2bc" (OuterVolumeSpecName: "kube-api-access-zv2bc") pod "947fe0f8-2b55-4a63-949a-4f20aa646f57" (UID: "947fe0f8-2b55-4a63-949a-4f20aa646f57"). InnerVolumeSpecName "kube-api-access-zv2bc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:18:19.228727 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.228642    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvn48\" (UniqueName: \"kubernetes.io/projected/d034ebda-561e-4dcb-bd56-60918690d7e6-kube-api-access-pvn48\") pod \"d034ebda-561e-4dcb-bd56-60918690d7e6\" (UID: \"d034ebda-561e-4dcb-bd56-60918690d7e6\") "
Jul 30 18:18:19.228727 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.228724    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d034ebda-561e-4dcb-bd56-60918690d7e6-plugins-config-volume\") pod \"d034ebda-561e-4dcb-bd56-60918690d7e6\" (UID: \"d034ebda-561e-4dcb-bd56-60918690d7e6\") "
Jul 30 18:18:19.228987 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.228926    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zv2bc\" (UniqueName: \"kubernetes.io/projected/947fe0f8-2b55-4a63-949a-4f20aa646f57-kube-api-access-zv2bc\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:18:19.228987 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.228938    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/947fe0f8-2b55-4a63-949a-4f20aa646f57-openshift-service-ca\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:18:19.228987 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.228948    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/947fe0f8-2b55-4a63-949a-4f20aa646f57-maas-api-tls\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:18:19.229181 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.229154    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d034ebda-561e-4dcb-bd56-60918690d7e6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d034ebda-561e-4dcb-bd56-60918690d7e6" (UID: "d034ebda-561e-4dcb-bd56-60918690d7e6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:18:19.230825 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.230793    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d034ebda-561e-4dcb-bd56-60918690d7e6-kube-api-access-pvn48" (OuterVolumeSpecName: "kube-api-access-pvn48") pod "d034ebda-561e-4dcb-bd56-60918690d7e6" (UID: "d034ebda-561e-4dcb-bd56-60918690d7e6"). InnerVolumeSpecName "kube-api-access-pvn48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:18:19.329915 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.329879    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvn48\" (UniqueName: \"kubernetes.io/projected/d034ebda-561e-4dcb-bd56-60918690d7e6-kube-api-access-pvn48\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:18:19.329915 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.329910    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d034ebda-561e-4dcb-bd56-60918690d7e6-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:18:19.331932 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.331905    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:19.397943 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.397910    2577 generic.go:356] "Generic (PLEG): container finished" podID="d034ebda-561e-4dcb-bd56-60918690d7e6" containerID="2cc5e3921724b37a4ddf3c5a4685d9a04a208233ea7af9cba2e4b1b84d819b42" exitCode=0
Jul 30 18:18:19.398150 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.397963    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz"
Jul 30 18:18:19.398150 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.397997    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz" event={"ID":"d034ebda-561e-4dcb-bd56-60918690d7e6","Type":"ContainerDied","Data":"2cc5e3921724b37a4ddf3c5a4685d9a04a208233ea7af9cba2e4b1b84d819b42"}
Jul 30 18:18:19.398150 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.398045    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz" event={"ID":"d034ebda-561e-4dcb-bd56-60918690d7e6","Type":"ContainerDied","Data":"7583a14b3a38aa7bfb4d17a03eddcfd32594e8d0d06f32325ae4a725730241bc"}
Jul 30 18:18:19.398150 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.398073    2577 scope.go:117] "RemoveContainer" containerID="2cc5e3921724b37a4ddf3c5a4685d9a04a208233ea7af9cba2e4b1b84d819b42"
Jul 30 18:18:19.399395 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.399372    2577 generic.go:356] "Generic (PLEG): container finished" podID="947fe0f8-2b55-4a63-949a-4f20aa646f57" containerID="8e3271d076c9cf09d4f59b9ff162161ffd765eb02222d2ef7b1c7bac429f6d1c" exitCode=0
Jul 30 18:18:19.399480 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.399406    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs"
Jul 30 18:18:19.399480 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.399448    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs" event={"ID":"947fe0f8-2b55-4a63-949a-4f20aa646f57","Type":"ContainerDied","Data":"8e3271d076c9cf09d4f59b9ff162161ffd765eb02222d2ef7b1c7bac429f6d1c"}
Jul 30 18:18:19.399602 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.399484    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs" event={"ID":"947fe0f8-2b55-4a63-949a-4f20aa646f57","Type":"ContainerDied","Data":"fcdd39f02a7b44482eba9fc566be53ff3ee81eead44d9df064fc3c3609ebf041"}
Jul 30 18:18:19.408467 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.408452    2577 scope.go:117] "RemoveContainer" containerID="2cc5e3921724b37a4ddf3c5a4685d9a04a208233ea7af9cba2e4b1b84d819b42"
Jul 30 18:18:19.408767 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:18:19.408749    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cc5e3921724b37a4ddf3c5a4685d9a04a208233ea7af9cba2e4b1b84d819b42\": container with ID starting with 2cc5e3921724b37a4ddf3c5a4685d9a04a208233ea7af9cba2e4b1b84d819b42 not found: ID does not exist" containerID="2cc5e3921724b37a4ddf3c5a4685d9a04a208233ea7af9cba2e4b1b84d819b42"
Jul 30 18:18:19.408869 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.408774    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cc5e3921724b37a4ddf3c5a4685d9a04a208233ea7af9cba2e4b1b84d819b42"} err="failed to get container status \"2cc5e3921724b37a4ddf3c5a4685d9a04a208233ea7af9cba2e4b1b84d819b42\": rpc error: code = NotFound desc = could not find container \"2cc5e3921724b37a4ddf3c5a4685d9a04a208233ea7af9cba2e4b1b84d819b42\": container with ID starting with 2cc5e3921724b37a4ddf3c5a4685d9a04a208233ea7af9cba2e4b1b84d819b42 not found: ID does not exist"
Jul 30 18:18:19.408869 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.408791    2577 scope.go:117] "RemoveContainer" containerID="8e3271d076c9cf09d4f59b9ff162161ffd765eb02222d2ef7b1c7bac429f6d1c"
Jul 30 18:18:19.417082 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.417041    2577 scope.go:117] "RemoveContainer" containerID="8e3271d076c9cf09d4f59b9ff162161ffd765eb02222d2ef7b1c7bac429f6d1c"
Jul 30 18:18:19.417403 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:18:19.417374    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e3271d076c9cf09d4f59b9ff162161ffd765eb02222d2ef7b1c7bac429f6d1c\": container with ID starting with 8e3271d076c9cf09d4f59b9ff162161ffd765eb02222d2ef7b1c7bac429f6d1c not found: ID does not exist" containerID="8e3271d076c9cf09d4f59b9ff162161ffd765eb02222d2ef7b1c7bac429f6d1c"
Jul 30 18:18:19.417507 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.417414    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e3271d076c9cf09d4f59b9ff162161ffd765eb02222d2ef7b1c7bac429f6d1c"} err="failed to get container status \"8e3271d076c9cf09d4f59b9ff162161ffd765eb02222d2ef7b1c7bac429f6d1c\": rpc error: code = NotFound desc = could not find container \"8e3271d076c9cf09d4f59b9ff162161ffd765eb02222d2ef7b1c7bac429f6d1c\": container with ID starting with 8e3271d076c9cf09d4f59b9ff162161ffd765eb02222d2ef7b1c7bac429f6d1c not found: ID does not exist"
Jul 30 18:18:19.419712 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.419691    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz"]
Jul 30 18:18:19.423449 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.423430    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b51ede-7487fd97b-pzvwz"]
Jul 30 18:18:19.432736 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.432716    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs"]
Jul 30 18:18:19.436425 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:19.436405    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b51ede-6dcd499c69-sg2bs"]
Jul 30 18:18:20.111650 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:20.111615    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="947fe0f8-2b55-4a63-949a-4f20aa646f57" path="/var/lib/kubelet/pods/947fe0f8-2b55-4a63-949a-4f20aa646f57/volumes"
Jul 30 18:18:20.112063 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:20.112043    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d034ebda-561e-4dcb-bd56-60918690d7e6" path="/var/lib/kubelet/pods/d034ebda-561e-4dcb-bd56-60918690d7e6/volumes"
Jul 30 18:18:25.373007 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:25.372973    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:25.462834 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:25.462636    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:25.611239 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:25.611205    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:25.848516 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:25.848483    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:26.194580 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:26.194487    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:26.530684 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:26.530647    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:26.690673 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:26.690636    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:29.725561 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:29.725527    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:29.813633 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:29.813598    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:30.841261 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:30.841231    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45"]
Jul 30 18:18:30.841700 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:30.841655    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8c5f5da-529b-4af1-bbd8-c56a974462ee" containerName="maas-api"
Jul 30 18:18:30.841700 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:30.841671    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8c5f5da-529b-4af1-bbd8-c56a974462ee" containerName="maas-api"
Jul 30 18:18:30.841700 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:30.841694    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="496f7c93-8b7c-429d-8a63-9ce26e160ddf" containerName="payload-pre-processing"
Jul 30 18:18:30.841950 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:30.841704    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="496f7c93-8b7c-429d-8a63-9ce26e160ddf" containerName="payload-pre-processing"
Jul 30 18:18:30.841950 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:30.841719    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="947fe0f8-2b55-4a63-949a-4f20aa646f57" containerName="maas-api"
Jul 30 18:18:30.841950 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:30.841729    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="947fe0f8-2b55-4a63-949a-4f20aa646f57" containerName="maas-api"
Jul 30 18:18:30.841950 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:30.841745    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d034ebda-561e-4dcb-bd56-60918690d7e6" containerName="payload-pre-processing"
Jul 30 18:18:30.841950 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:30.841754    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d034ebda-561e-4dcb-bd56-60918690d7e6" containerName="payload-pre-processing"
Jul 30 18:18:30.841950 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:30.841787    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e9d3ec1-7ee1-4615-820c-415c02ce5600" containerName="payload-processing"
Jul 30 18:18:30.841950 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:30.841797    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e9d3ec1-7ee1-4615-820c-415c02ce5600" containerName="payload-processing"
Jul 30 18:18:30.841950 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:30.841947    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e9d3ec1-7ee1-4615-820c-415c02ce5600" containerName="payload-processing"
Jul 30 18:18:30.842345 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:30.841961    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="947fe0f8-2b55-4a63-949a-4f20aa646f57" containerName="maas-api"
Jul 30 18:18:30.842345 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:30.841974    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d034ebda-561e-4dcb-bd56-60918690d7e6" containerName="payload-pre-processing"
Jul 30 18:18:30.842345 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:30.841984    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="496f7c93-8b7c-429d-8a63-9ce26e160ddf" containerName="payload-pre-processing"
Jul 30 18:18:30.842345 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:30.841991    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8c5f5da-529b-4af1-bbd8-c56a974462ee" containerName="maas-api"
Jul 30 18:18:30.845319 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:30.845301    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45"
Jul 30 18:18:30.847521 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:30.847500    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-8f586d-serving-cert\""
Jul 30 18:18:30.854870 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:30.854798    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45"]
Jul 30 18:18:30.870595 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:30.870569    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:30.940459 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:30.940425    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15fc6fff-6986-4a20-96c8-ecdc35841c21-maas-api-tls\") pod \"maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45\" (UID: \"15fc6fff-6986-4a20-96c8-ecdc35841c21\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45"
Jul 30 18:18:30.940623 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:30.940481    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9b6l\" (UniqueName: \"kubernetes.io/projected/15fc6fff-6986-4a20-96c8-ecdc35841c21-kube-api-access-j9b6l\") pod \"maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45\" (UID: \"15fc6fff-6986-4a20-96c8-ecdc35841c21\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45"
Jul 30 18:18:30.940623 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:30.940539    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15fc6fff-6986-4a20-96c8-ecdc35841c21-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45\" (UID: \"15fc6fff-6986-4a20-96c8-ecdc35841c21\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45"
Jul 30 18:18:31.029746 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.029698    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:31.041384 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.041359    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15fc6fff-6986-4a20-96c8-ecdc35841c21-maas-api-tls\") pod \"maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45\" (UID: \"15fc6fff-6986-4a20-96c8-ecdc35841c21\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45"
Jul 30 18:18:31.041493 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.041408    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9b6l\" (UniqueName: \"kubernetes.io/projected/15fc6fff-6986-4a20-96c8-ecdc35841c21-kube-api-access-j9b6l\") pod \"maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45\" (UID: \"15fc6fff-6986-4a20-96c8-ecdc35841c21\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45"
Jul 30 18:18:31.041493 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.041454    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15fc6fff-6986-4a20-96c8-ecdc35841c21-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45\" (UID: \"15fc6fff-6986-4a20-96c8-ecdc35841c21\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45"
Jul 30 18:18:31.042176 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.042108    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15fc6fff-6986-4a20-96c8-ecdc35841c21-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45\" (UID: \"15fc6fff-6986-4a20-96c8-ecdc35841c21\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45"
Jul 30 18:18:31.043972 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.043945    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15fc6fff-6986-4a20-96c8-ecdc35841c21-maas-api-tls\") pod \"maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45\" (UID: \"15fc6fff-6986-4a20-96c8-ecdc35841c21\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45"
Jul 30 18:18:31.049914 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.049889    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9b6l\" (UniqueName: \"kubernetes.io/projected/15fc6fff-6986-4a20-96c8-ecdc35841c21-kube-api-access-j9b6l\") pod \"maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45\" (UID: \"15fc6fff-6986-4a20-96c8-ecdc35841c21\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45"
Jul 30 18:18:31.156848 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.156731    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45"
Jul 30 18:18:31.302524 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.302503    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45"]
Jul 30 18:18:31.305067 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:18:31.305034    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15fc6fff_6986_4a20_96c8_ecdc35841c21.slice/crio-1bad3ad529fe91d7fa4cd0b01e9a80eecb0fdc58e9c2383a8cca8a415548e03d WatchSource:0}: Error finding container 1bad3ad529fe91d7fa4cd0b01e9a80eecb0fdc58e9c2383a8cca8a415548e03d: Status 404 returned error can't find the container with id 1bad3ad529fe91d7fa4cd0b01e9a80eecb0fdc58e9c2383a8cca8a415548e03d
Jul 30 18:18:31.468424 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.468337    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45" event={"ID":"15fc6fff-6986-4a20-96c8-ecdc35841c21","Type":"ContainerStarted","Data":"1bad3ad529fe91d7fa4cd0b01e9a80eecb0fdc58e9c2383a8cca8a415548e03d"}
Jul 30 18:18:31.481604 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.481574    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4"]
Jul 30 18:18:31.486607 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.486581    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4"
Jul 30 18:18:31.488988 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.488956    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-8f586d\""
Jul 30 18:18:31.488988 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.488967    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-8f586d-dockercfg-pbb8w\""
Jul 30 18:18:31.494484 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.494451    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4"]
Jul 30 18:18:31.501996 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.501967    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb"]
Jul 30 18:18:31.505484 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.505462    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb"
Jul 30 18:18:31.518563 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.518524    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb"]
Jul 30 18:18:31.545440 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.545411    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/925d2bea-64eb-4227-9b5a-8467d7507d92-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4\" (UID: \"925d2bea-64eb-4227-9b5a-8467d7507d92\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4"
Jul 30 18:18:31.545613 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.545474    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7whf\" (UniqueName: \"kubernetes.io/projected/9bf79219-bc3f-4e41-a044-f1f790051406-kube-api-access-p7whf\") pod \"payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb\" (UID: \"9bf79219-bc3f-4e41-a044-f1f790051406\") " pod="openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb"
Jul 30 18:18:31.545613 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.545548    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhzxq\" (UniqueName: \"kubernetes.io/projected/925d2bea-64eb-4227-9b5a-8467d7507d92-kube-api-access-hhzxq\") pod \"payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4\" (UID: \"925d2bea-64eb-4227-9b5a-8467d7507d92\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4"
Jul 30 18:18:31.545709 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.545640    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bf79219-bc3f-4e41-a044-f1f790051406-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb\" (UID: \"9bf79219-bc3f-4e41-a044-f1f790051406\") " pod="openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb"
Jul 30 18:18:31.646746 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.646693    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhzxq\" (UniqueName: \"kubernetes.io/projected/925d2bea-64eb-4227-9b5a-8467d7507d92-kube-api-access-hhzxq\") pod \"payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4\" (UID: \"925d2bea-64eb-4227-9b5a-8467d7507d92\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4"
Jul 30 18:18:31.646966 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.646790    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bf79219-bc3f-4e41-a044-f1f790051406-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb\" (UID: \"9bf79219-bc3f-4e41-a044-f1f790051406\") " pod="openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb"
Jul 30 18:18:31.646966 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.646863    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/925d2bea-64eb-4227-9b5a-8467d7507d92-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4\" (UID: \"925d2bea-64eb-4227-9b5a-8467d7507d92\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4"
Jul 30 18:18:31.646966 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.646942    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7whf\" (UniqueName: \"kubernetes.io/projected/9bf79219-bc3f-4e41-a044-f1f790051406-kube-api-access-p7whf\") pod \"payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb\" (UID: \"9bf79219-bc3f-4e41-a044-f1f790051406\") " pod="openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb"
Jul 30 18:18:31.647551 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.647531    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/925d2bea-64eb-4227-9b5a-8467d7507d92-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4\" (UID: \"925d2bea-64eb-4227-9b5a-8467d7507d92\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4"
Jul 30 18:18:31.647590 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.647540    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bf79219-bc3f-4e41-a044-f1f790051406-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb\" (UID: \"9bf79219-bc3f-4e41-a044-f1f790051406\") " pod="openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb"
Jul 30 18:18:31.655237 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.655204    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhzxq\" (UniqueName: \"kubernetes.io/projected/925d2bea-64eb-4227-9b5a-8467d7507d92-kube-api-access-hhzxq\") pod \"payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4\" (UID: \"925d2bea-64eb-4227-9b5a-8467d7507d92\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4"
Jul 30 18:18:31.655364 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.655201    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7whf\" (UniqueName: \"kubernetes.io/projected/9bf79219-bc3f-4e41-a044-f1f790051406-kube-api-access-p7whf\") pod \"payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb\" (UID: \"9bf79219-bc3f-4e41-a044-f1f790051406\") " pod="openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb"
Jul 30 18:18:31.659835 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.659785    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:31.783204 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.783153    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:31.799148 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.799114    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4"
Jul 30 18:18:31.824406 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.824378    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb"
Jul 30 18:18:31.958247 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.956919    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4"]
Jul 30 18:18:31.958647 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:18:31.958332    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod925d2bea_64eb_4227_9b5a_8467d7507d92.slice/crio-b7b36acd3981f1bfceed90eafec9d0e70788ac74c97ad58f9a9e3cc91ae074ad WatchSource:0}: Error finding container b7b36acd3981f1bfceed90eafec9d0e70788ac74c97ad58f9a9e3cc91ae074ad: Status 404 returned error can't find the container with id b7b36acd3981f1bfceed90eafec9d0e70788ac74c97ad58f9a9e3cc91ae074ad
Jul 30 18:18:31.984741 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:31.984713    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb"]
Jul 30 18:18:31.999155 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:18:31.999122    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bf79219_bc3f_4e41_a044_f1f790051406.slice/crio-1fdcf2e49f5c2261f1e96a2a27ebedb74ba33e788dae6ff2d5d4194a103e34b6 WatchSource:0}: Error finding container 1fdcf2e49f5c2261f1e96a2a27ebedb74ba33e788dae6ff2d5d4194a103e34b6: Status 404 returned error can't find the container with id 1fdcf2e49f5c2261f1e96a2a27ebedb74ba33e788dae6ff2d5d4194a103e34b6
Jul 30 18:18:32.477110 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:32.477077    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4" event={"ID":"925d2bea-64eb-4227-9b5a-8467d7507d92","Type":"ContainerStarted","Data":"aad44f32d189a0d90952d9eba5b4818c65c13e85d4cdacc0d63492d3779d80a1"}
Jul 30 18:18:32.477290 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:32.477129    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4" event={"ID":"925d2bea-64eb-4227-9b5a-8467d7507d92","Type":"ContainerStarted","Data":"b7b36acd3981f1bfceed90eafec9d0e70788ac74c97ad58f9a9e3cc91ae074ad"}
Jul 30 18:18:32.477290 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:32.477174    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4"
Jul 30 18:18:32.478803 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:32.478778    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45" event={"ID":"15fc6fff-6986-4a20-96c8-ecdc35841c21","Type":"ContainerStarted","Data":"d3b64eb3e74ba23db4572fedd7a6614bc2c1c5f204458f692bd09a214ed74005"}
Jul 30 18:18:32.478970 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:32.478874    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45"
Jul 30 18:18:32.480396 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:32.480371    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb" event={"ID":"9bf79219-bc3f-4e41-a044-f1f790051406","Type":"ContainerStarted","Data":"54e858b6ca216bea68ead3230fc5cc312d1c97cefbde2e24e351cf5a7129bdbe"}
Jul 30 18:18:32.480509 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:32.480406    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb" event={"ID":"9bf79219-bc3f-4e41-a044-f1f790051406","Type":"ContainerStarted","Data":"1fdcf2e49f5c2261f1e96a2a27ebedb74ba33e788dae6ff2d5d4194a103e34b6"}
Jul 30 18:18:32.480594 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:32.480505    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb"
Jul 30 18:18:32.516464 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:32.516416    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4" podStartSLOduration=1.516400706 podStartE2EDuration="1.516400706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:18:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:18:32.515457528 +0000 UTC m=+3086.969021519" watchObservedRunningTime="2026-07-30 18:18:32.516400706 +0000 UTC m=+3086.969964696"
Jul 30 18:18:32.548142 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:32.548094    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb" podStartSLOduration=1.548076575 podStartE2EDuration="1.548076575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:18:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:18:32.545270824 +0000 UTC m=+3086.998834814" watchObservedRunningTime="2026-07-30 18:18:32.548076575 +0000 UTC m=+3087.001640567"
Jul 30 18:18:32.566683 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:32.566629    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45" podStartSLOduration=2.174014705 podStartE2EDuration="2.566612222s" totalImagesPullingTime="392.597517ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:18:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:18:32.564783064 +0000 UTC m=+3087.018347057" watchObservedRunningTime="2026-07-30 18:18:32.566612222 +0000 UTC m=+3087.020176213"
Jul 30 18:18:38.491131 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:38.491102    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45"
Jul 30 18:18:43.488467 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:43.488429    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb"
Jul 30 18:18:43.488873 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:43.488591    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4"
Jul 30 18:18:54.765983 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:54.765951    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:55.107721 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:55.107611    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:55.430377 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:55.430266    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:55.587227 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:55.587190    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:58.873702 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:58.873667    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:58.973619 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:58.973573    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:18:59.966083 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:59.966048    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg"]
Jul 30 18:18:59.970487 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:59.970464    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg"
Jul 30 18:18:59.972700 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:59.972680    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-fbbaa9-serving-cert\""
Jul 30 18:18:59.980793 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:18:59.980713    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg"]
Jul 30 18:19:00.130322 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.130281    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96m75\" (UniqueName: \"kubernetes.io/projected/b9518ffe-8b29-4784-854e-3a31b71e8624-kube-api-access-96m75\") pod \"maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg\" (UID: \"b9518ffe-8b29-4784-854e-3a31b71e8624\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg"
Jul 30 18:19:00.130512 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.130344    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9518ffe-8b29-4784-854e-3a31b71e8624-maas-api-tls\") pod \"maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg\" (UID: \"b9518ffe-8b29-4784-854e-3a31b71e8624\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg"
Jul 30 18:19:00.130512 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.130370    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9518ffe-8b29-4784-854e-3a31b71e8624-openshift-service-ca\") pod \"maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg\" (UID: \"b9518ffe-8b29-4784-854e-3a31b71e8624\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg"
Jul 30 18:19:00.173045 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.173015    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:19:00.231564 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.231481    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96m75\" (UniqueName: \"kubernetes.io/projected/b9518ffe-8b29-4784-854e-3a31b71e8624-kube-api-access-96m75\") pod \"maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg\" (UID: \"b9518ffe-8b29-4784-854e-3a31b71e8624\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg"
Jul 30 18:19:00.231564 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.231529    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9518ffe-8b29-4784-854e-3a31b71e8624-maas-api-tls\") pod \"maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg\" (UID: \"b9518ffe-8b29-4784-854e-3a31b71e8624\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg"
Jul 30 18:19:00.231564 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.231548    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9518ffe-8b29-4784-854e-3a31b71e8624-openshift-service-ca\") pod \"maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg\" (UID: \"b9518ffe-8b29-4784-854e-3a31b71e8624\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg"
Jul 30 18:19:00.232176 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.232156    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9518ffe-8b29-4784-854e-3a31b71e8624-openshift-service-ca\") pod \"maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg\" (UID: \"b9518ffe-8b29-4784-854e-3a31b71e8624\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg"
Jul 30 18:19:00.234344 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.234314    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9518ffe-8b29-4784-854e-3a31b71e8624-maas-api-tls\") pod \"maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg\" (UID: \"b9518ffe-8b29-4784-854e-3a31b71e8624\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg"
Jul 30 18:19:00.239183 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.239166    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96m75\" (UniqueName: \"kubernetes.io/projected/b9518ffe-8b29-4784-854e-3a31b71e8624-kube-api-access-96m75\") pod \"maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg\" (UID: \"b9518ffe-8b29-4784-854e-3a31b71e8624\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg"
Jul 30 18:19:00.274106 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.274071    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:19:00.284227 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.284206    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg"
Jul 30 18:19:00.430135 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.430112    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg"]
Jul 30 18:19:00.432277 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:19:00.432249    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9518ffe_8b29_4784_854e_3a31b71e8624.slice/crio-3f31fba30bbabbc8fd9aa37fc464c95b51543992e411371a56e9dd16a128c625 WatchSource:0}: Error finding container 3f31fba30bbabbc8fd9aa37fc464c95b51543992e411371a56e9dd16a128c625: Status 404 returned error can't find the container with id 3f31fba30bbabbc8fd9aa37fc464c95b51543992e411371a56e9dd16a128c625
Jul 30 18:19:00.485847 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.485713    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:19:00.582140 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.582093    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:19:00.582644 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.582623    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp"]
Jul 30 18:19:00.587564 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.587544    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp"
Jul 30 18:19:00.589646 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.589628    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-fbbaa9\""
Jul 30 18:19:00.596981 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.596960    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp"]
Jul 30 18:19:00.640526 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.640491    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg" event={"ID":"b9518ffe-8b29-4784-854e-3a31b71e8624","Type":"ContainerStarted","Data":"3f31fba30bbabbc8fd9aa37fc464c95b51543992e411371a56e9dd16a128c625"}
Jul 30 18:19:00.736031 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.735937    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwrjd\" (UniqueName: \"kubernetes.io/projected/5dc55636-aca9-48d8-8ba1-8c6110c262f7-kube-api-access-hwrjd\") pod \"payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp\" (UID: \"5dc55636-aca9-48d8-8ba1-8c6110c262f7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp"
Jul 30 18:19:00.736210 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.736055    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5dc55636-aca9-48d8-8ba1-8c6110c262f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp\" (UID: \"5dc55636-aca9-48d8-8ba1-8c6110c262f7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp"
Jul 30 18:19:00.837015 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.836990    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwrjd\" (UniqueName: \"kubernetes.io/projected/5dc55636-aca9-48d8-8ba1-8c6110c262f7-kube-api-access-hwrjd\") pod \"payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp\" (UID: \"5dc55636-aca9-48d8-8ba1-8c6110c262f7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp"
Jul 30 18:19:00.837134 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.837064    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5dc55636-aca9-48d8-8ba1-8c6110c262f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp\" (UID: \"5dc55636-aca9-48d8-8ba1-8c6110c262f7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp"
Jul 30 18:19:00.837685 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.837665    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5dc55636-aca9-48d8-8ba1-8c6110c262f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp\" (UID: \"5dc55636-aca9-48d8-8ba1-8c6110c262f7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp"
Jul 30 18:19:00.845907 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.845884    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwrjd\" (UniqueName: \"kubernetes.io/projected/5dc55636-aca9-48d8-8ba1-8c6110c262f7-kube-api-access-hwrjd\") pod \"payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp\" (UID: \"5dc55636-aca9-48d8-8ba1-8c6110c262f7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp"
Jul 30 18:19:00.898943 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:00.898907    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp"
Jul 30 18:19:01.043941 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:01.043870    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp"]
Jul 30 18:19:01.046287 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:19:01.046258    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dc55636_aca9_48d8_8ba1_8c6110c262f7.slice/crio-36f0c107ca4775a6f031db5d799b2b0326549db7dc9e52da4ac975d54e99382d WatchSource:0}: Error finding container 36f0c107ca4775a6f031db5d799b2b0326549db7dc9e52da4ac975d54e99382d: Status 404 returned error can't find the container with id 36f0c107ca4775a6f031db5d799b2b0326549db7dc9e52da4ac975d54e99382d
Jul 30 18:19:01.650407 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:01.650362    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp" event={"ID":"5dc55636-aca9-48d8-8ba1-8c6110c262f7","Type":"ContainerStarted","Data":"6b3bdd2aa6a6227e3a1cd700b2a48403253ded894f4ada49c697903752f555b2"}
Jul 30 18:19:01.650618 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:01.650428    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp"
Jul 30 18:19:01.650618 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:01.650446    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp" event={"ID":"5dc55636-aca9-48d8-8ba1-8c6110c262f7","Type":"ContainerStarted","Data":"36f0c107ca4775a6f031db5d799b2b0326549db7dc9e52da4ac975d54e99382d"}
Jul 30 18:19:01.652142 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:01.652117    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg" event={"ID":"b9518ffe-8b29-4784-854e-3a31b71e8624","Type":"ContainerStarted","Data":"fa047225b144e5115ae1923ca4360bb960263ddacd2078728376f4b8bc22782d"}
Jul 30 18:19:01.652259 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:01.652213    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg"
Jul 30 18:19:01.666246 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:01.666182    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp" podStartSLOduration=1.6661616769999998 podStartE2EDuration="1.666161677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:19:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:19:01.665610242 +0000 UTC m=+3116.119174241" watchObservedRunningTime="2026-07-30 18:19:01.666161677 +0000 UTC m=+3116.119725668"
Jul 30 18:19:01.681738 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:01.681683    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg" podStartSLOduration=2.31024399 podStartE2EDuration="2.681664674s" totalImagesPullingTime="371.420684ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:18:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:19:01.680774299 +0000 UTC m=+3116.134338291" watchObservedRunningTime="2026-07-30 18:19:01.681664674 +0000 UTC m=+3116.135228664"
Jul 30 18:19:07.663397 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:07.663360    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg"
Jul 30 18:19:12.660096 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:12.660062    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp"
Jul 30 18:19:23.932881 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:23.926784    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:19:28.246284 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:28.246258    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:19:54.153964 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:19:54.153926    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:20:44.538709 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:20:44.538673    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:22:12.125005 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:22:12.124967    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:22:56.219340 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:22:56.219305    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:22:57.737860 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:22:57.737804    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:23:06.520461 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:06.520429    2577 scope.go:117] "RemoveContainer" containerID="4a567ff58a4f3ba61b3bd87c536176f3f0e8f889881e5848035b30bad54fe901"
Jul 30 18:23:10.924655 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:10.924617    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jvwv9"]
Jul 30 18:23:10.929787 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:10.929766    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jvwv9"
Jul 30 18:23:10.946104 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:10.946078    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jvwv9"]
Jul 30 18:23:10.979882 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:10.979848    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzhdl\" (UniqueName: \"kubernetes.io/projected/8dfb9ab0-36fe-409d-a562-91f97c6724d4-kube-api-access-gzhdl\") pod \"kuadrant-operator-catalog-jvwv9\" (UID: \"8dfb9ab0-36fe-409d-a562-91f97c6724d4\") " pod="kuadrant-system/kuadrant-operator-catalog-jvwv9"
Jul 30 18:23:11.080464 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:11.080430    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzhdl\" (UniqueName: \"kubernetes.io/projected/8dfb9ab0-36fe-409d-a562-91f97c6724d4-kube-api-access-gzhdl\") pod \"kuadrant-operator-catalog-jvwv9\" (UID: \"8dfb9ab0-36fe-409d-a562-91f97c6724d4\") " pod="kuadrant-system/kuadrant-operator-catalog-jvwv9"
Jul 30 18:23:11.089433 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:11.089397    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzhdl\" (UniqueName: \"kubernetes.io/projected/8dfb9ab0-36fe-409d-a562-91f97c6724d4-kube-api-access-gzhdl\") pod \"kuadrant-operator-catalog-jvwv9\" (UID: \"8dfb9ab0-36fe-409d-a562-91f97c6724d4\") " pod="kuadrant-system/kuadrant-operator-catalog-jvwv9"
Jul 30 18:23:11.240213 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:11.240124    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jvwv9"
Jul 30 18:23:11.388607 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:11.388582    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jvwv9"]
Jul 30 18:23:11.390950 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:23:11.390910    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dfb9ab0_36fe_409d_a562_91f97c6724d4.slice/crio-133c09acfc42aeba576db4ec719c51519550e0d8bc41770070881421e59b5470 WatchSource:0}: Error finding container 133c09acfc42aeba576db4ec719c51519550e0d8bc41770070881421e59b5470: Status 404 returned error can't find the container with id 133c09acfc42aeba576db4ec719c51519550e0d8bc41770070881421e59b5470
Jul 30 18:23:11.392447 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:11.392426    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 18:23:12.017463 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:12.017428    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jvwv9" event={"ID":"8dfb9ab0-36fe-409d-a562-91f97c6724d4","Type":"ContainerStarted","Data":"2f929fe21762be6e4c1756415bbc1bd6e51da16d79251880776a52f3b39d0795"}
Jul 30 18:23:12.017890 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:12.017483    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jvwv9" event={"ID":"8dfb9ab0-36fe-409d-a562-91f97c6724d4","Type":"ContainerStarted","Data":"133c09acfc42aeba576db4ec719c51519550e0d8bc41770070881421e59b5470"}
Jul 30 18:23:12.033607 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:12.033511    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-jvwv9" podStartSLOduration=1.656074756 podStartE2EDuration="2.033496736s" totalImagesPullingTime="377.42198ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:23:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:23:12.032388695 +0000 UTC m=+3366.485952688" watchObservedRunningTime="2026-07-30 18:23:12.033496736 +0000 UTC m=+3366.487060727"
Jul 30 18:23:21.240618 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:21.240538    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-jvwv9"
Jul 30 18:23:21.240618 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:21.240601    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-jvwv9"
Jul 30 18:23:21.263233 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:21.263207    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-jvwv9"
Jul 30 18:23:22.091102 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:22.091075    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-jvwv9"
Jul 30 18:23:22.123387 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:22.123351    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jvwv9"]
Jul 30 18:23:24.078616 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:24.078580    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-jvwv9" podUID="8dfb9ab0-36fe-409d-a562-91f97c6724d4" containerName="registry-server" containerID="cri-o://2f929fe21762be6e4c1756415bbc1bd6e51da16d79251880776a52f3b39d0795" gracePeriod=2
Jul 30 18:23:24.354092 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:24.354066    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jvwv9"
Jul 30 18:23:24.511495 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:24.511444    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzhdl\" (UniqueName: \"kubernetes.io/projected/8dfb9ab0-36fe-409d-a562-91f97c6724d4-kube-api-access-gzhdl\") pod \"8dfb9ab0-36fe-409d-a562-91f97c6724d4\" (UID: \"8dfb9ab0-36fe-409d-a562-91f97c6724d4\") "
Jul 30 18:23:24.513678 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:24.513640    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dfb9ab0-36fe-409d-a562-91f97c6724d4-kube-api-access-gzhdl" (OuterVolumeSpecName: "kube-api-access-gzhdl") pod "8dfb9ab0-36fe-409d-a562-91f97c6724d4" (UID: "8dfb9ab0-36fe-409d-a562-91f97c6724d4"). InnerVolumeSpecName "kube-api-access-gzhdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:23:24.612573 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:24.612484    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzhdl\" (UniqueName: \"kubernetes.io/projected/8dfb9ab0-36fe-409d-a562-91f97c6724d4-kube-api-access-gzhdl\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:23:25.085267 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:25.085231    2577 generic.go:356] "Generic (PLEG): container finished" podID="8dfb9ab0-36fe-409d-a562-91f97c6724d4" containerID="2f929fe21762be6e4c1756415bbc1bd6e51da16d79251880776a52f3b39d0795" exitCode=0
Jul 30 18:23:25.085675 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:25.085278    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jvwv9"
Jul 30 18:23:25.085675 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:25.085319    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jvwv9" event={"ID":"8dfb9ab0-36fe-409d-a562-91f97c6724d4","Type":"ContainerDied","Data":"2f929fe21762be6e4c1756415bbc1bd6e51da16d79251880776a52f3b39d0795"}
Jul 30 18:23:25.085675 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:25.085400    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jvwv9" event={"ID":"8dfb9ab0-36fe-409d-a562-91f97c6724d4","Type":"ContainerDied","Data":"133c09acfc42aeba576db4ec719c51519550e0d8bc41770070881421e59b5470"}
Jul 30 18:23:25.085675 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:25.085427    2577 scope.go:117] "RemoveContainer" containerID="2f929fe21762be6e4c1756415bbc1bd6e51da16d79251880776a52f3b39d0795"
Jul 30 18:23:25.095620 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:25.095604    2577 scope.go:117] "RemoveContainer" containerID="2f929fe21762be6e4c1756415bbc1bd6e51da16d79251880776a52f3b39d0795"
Jul 30 18:23:25.095934 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:23:25.095909    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f929fe21762be6e4c1756415bbc1bd6e51da16d79251880776a52f3b39d0795\": container with ID starting with 2f929fe21762be6e4c1756415bbc1bd6e51da16d79251880776a52f3b39d0795 not found: ID does not exist" containerID="2f929fe21762be6e4c1756415bbc1bd6e51da16d79251880776a52f3b39d0795"
Jul 30 18:23:25.095985 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:25.095944    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f929fe21762be6e4c1756415bbc1bd6e51da16d79251880776a52f3b39d0795"} err="failed to get container status \"2f929fe21762be6e4c1756415bbc1bd6e51da16d79251880776a52f3b39d0795\": rpc error: code = NotFound desc = could not find container \"2f929fe21762be6e4c1756415bbc1bd6e51da16d79251880776a52f3b39d0795\": container with ID starting with 2f929fe21762be6e4c1756415bbc1bd6e51da16d79251880776a52f3b39d0795 not found: ID does not exist"
Jul 30 18:23:25.105624 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:25.105603    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jvwv9"]
Jul 30 18:23:25.108701 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:25.108676    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jvwv9"]
Jul 30 18:23:26.111336 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:26.111302    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dfb9ab0-36fe-409d-a562-91f97c6724d4" path="/var/lib/kubelet/pods/8dfb9ab0-36fe-409d-a562-91f97c6724d4/volumes"
Jul 30 18:23:45.114343 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:23:45.114304    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:24:05.689080 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:24:05.688978    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:24:46.897283 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:24:46.897249    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:24:58.031671 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:24:58.031631    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:24:59.099199 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:24:59.099163    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:25:07.626122 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:25:07.626091    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:25:30.333738 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:25:30.333632    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:27:10.146535 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:27:10.146496    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:27:11.045186 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:27:11.045160    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:27:15.037665 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:27:15.037633    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:27:39.811723 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:27:39.811681    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:29:29.065171 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:29.065115    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:29:30.896313 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:30.896276    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:29:31.080853 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:31.080797    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7426-140152180297264"]
Jul 30 18:29:31.081323 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:31.081308    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dfb9ab0-36fe-409d-a562-91f97c6724d4" containerName="registry-server"
Jul 30 18:29:31.081369 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:31.081325    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dfb9ab0-36fe-409d-a562-91f97c6724d4" containerName="registry-server"
Jul 30 18:29:31.081423 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:31.081414    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dfb9ab0-36fe-409d-a562-91f97c6724d4" containerName="registry-server"
Jul 30 18:29:31.084666 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:31.084646    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180297264"
Jul 30 18:29:31.086947 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:31.086914    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-87k6q\""
Jul 30 18:29:31.090888 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:31.090862    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140152180297264"]
Jul 30 18:29:31.226086 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:31.225974    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfqnt\" (UniqueName: \"kubernetes.io/projected/3ad4e9b5-495e-4c92-a0eb-cf6cde03615c-kube-api-access-lfqnt\") pod \"test-curl-7426-140152180297264\" (UID: \"3ad4e9b5-495e-4c92-a0eb-cf6cde03615c\") " pod="opendatahub/test-curl-7426-140152180297264"
Jul 30 18:29:31.327594 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:31.327546    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfqnt\" (UniqueName: \"kubernetes.io/projected/3ad4e9b5-495e-4c92-a0eb-cf6cde03615c-kube-api-access-lfqnt\") pod \"test-curl-7426-140152180297264\" (UID: \"3ad4e9b5-495e-4c92-a0eb-cf6cde03615c\") " pod="opendatahub/test-curl-7426-140152180297264"
Jul 30 18:29:31.336290 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:31.336264    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfqnt\" (UniqueName: \"kubernetes.io/projected/3ad4e9b5-495e-4c92-a0eb-cf6cde03615c-kube-api-access-lfqnt\") pod \"test-curl-7426-140152180297264\" (UID: \"3ad4e9b5-495e-4c92-a0eb-cf6cde03615c\") " pod="opendatahub/test-curl-7426-140152180297264"
Jul 30 18:29:31.395473 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:31.395437    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180297264"
Jul 30 18:29:31.540625 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:31.540100    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140152180297264"]
Jul 30 18:29:31.542787 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:29:31.542756    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ad4e9b5_495e_4c92_a0eb_cf6cde03615c.slice/crio-d17a82debd0d9637607d9745a9057612e902ce638dfd10fdf9274f387ec9ab2f WatchSource:0}: Error finding container d17a82debd0d9637607d9745a9057612e902ce638dfd10fdf9274f387ec9ab2f: Status 404 returned error can't find the container with id d17a82debd0d9637607d9745a9057612e902ce638dfd10fdf9274f387ec9ab2f
Jul 30 18:29:31.544432 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:31.544410    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 18:29:32.084292 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:32.084253    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140152180297264" event={"ID":"3ad4e9b5-495e-4c92-a0eb-cf6cde03615c","Type":"ContainerStarted","Data":"d17a82debd0d9637607d9745a9057612e902ce638dfd10fdf9274f387ec9ab2f"}
Jul 30 18:29:33.091092 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:33.091060    2577 generic.go:356] "Generic (PLEG): container finished" podID="3ad4e9b5-495e-4c92-a0eb-cf6cde03615c" containerID="827e82eb7829c58a3c820964578eb47d2d14a85491a027c92677869f5cdfafd9" exitCode=0
Jul 30 18:29:33.091492 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:33.091112    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140152180297264" event={"ID":"3ad4e9b5-495e-4c92-a0eb-cf6cde03615c","Type":"ContainerDied","Data":"827e82eb7829c58a3c820964578eb47d2d14a85491a027c92677869f5cdfafd9"}
Jul 30 18:29:34.241291 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:34.241262    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180297264"
Jul 30 18:29:34.263082 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:34.263049    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7426-140152180297264_3ad4e9b5-495e-4c92-a0eb-cf6cde03615c/test-curl-7426-140152180297264/0.log"
Jul 30 18:29:34.290044 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:34.290005    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7426-140152180297264"]
Jul 30 18:29:34.294690 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:34.294658    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7426-140152180297264"]
Jul 30 18:29:34.357369 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:34.357284    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfqnt\" (UniqueName: \"kubernetes.io/projected/3ad4e9b5-495e-4c92-a0eb-cf6cde03615c-kube-api-access-lfqnt\") pod \"3ad4e9b5-495e-4c92-a0eb-cf6cde03615c\" (UID: \"3ad4e9b5-495e-4c92-a0eb-cf6cde03615c\") "
Jul 30 18:29:34.359662 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:34.359630    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ad4e9b5-495e-4c92-a0eb-cf6cde03615c-kube-api-access-lfqnt" (OuterVolumeSpecName: "kube-api-access-lfqnt") pod "3ad4e9b5-495e-4c92-a0eb-cf6cde03615c" (UID: "3ad4e9b5-495e-4c92-a0eb-cf6cde03615c"). InnerVolumeSpecName "kube-api-access-lfqnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:29:34.423740 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:34.423703    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7426-140152180281904"]
Jul 30 18:29:34.424182 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:34.424167    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ad4e9b5-495e-4c92-a0eb-cf6cde03615c" containerName="test-curl-7426-140152180297264"
Jul 30 18:29:34.424235 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:34.424184    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ad4e9b5-495e-4c92-a0eb-cf6cde03615c" containerName="test-curl-7426-140152180297264"
Jul 30 18:29:34.424273 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:34.424259    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ad4e9b5-495e-4c92-a0eb-cf6cde03615c" containerName="test-curl-7426-140152180297264"
Jul 30 18:29:34.427558 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:34.427543    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180281904"
Jul 30 18:29:34.433375 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:34.433345    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140152180281904"]
Jul 30 18:29:34.458895 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:34.458838    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfqnt\" (UniqueName: \"kubernetes.io/projected/3ad4e9b5-495e-4c92-a0eb-cf6cde03615c-kube-api-access-lfqnt\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:29:34.559740 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:34.559704    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhhjw\" (UniqueName: \"kubernetes.io/projected/98b3c790-eee0-4983-a49f-633122c2605b-kube-api-access-xhhjw\") pod \"test-curl-7426-140152180281904\" (UID: \"98b3c790-eee0-4983-a49f-633122c2605b\") " pod="opendatahub/test-curl-7426-140152180281904"
Jul 30 18:29:34.661282 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:34.661186    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhhjw\" (UniqueName: \"kubernetes.io/projected/98b3c790-eee0-4983-a49f-633122c2605b-kube-api-access-xhhjw\") pod \"test-curl-7426-140152180281904\" (UID: \"98b3c790-eee0-4983-a49f-633122c2605b\") " pod="opendatahub/test-curl-7426-140152180281904"
Jul 30 18:29:34.669734 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:34.669706    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhhjw\" (UniqueName: \"kubernetes.io/projected/98b3c790-eee0-4983-a49f-633122c2605b-kube-api-access-xhhjw\") pod \"test-curl-7426-140152180281904\" (UID: \"98b3c790-eee0-4983-a49f-633122c2605b\") " pod="opendatahub/test-curl-7426-140152180281904"
Jul 30 18:29:34.738937 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:34.738879    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180281904"
Jul 30 18:29:34.882026 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:34.881994    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140152180281904"]
Jul 30 18:29:34.884410 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:29:34.884368    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98b3c790_eee0_4983_a49f_633122c2605b.slice/crio-27e07b3b398b41efc61eee6cc12e443c798f41d839d1ef5350894fea09232ec3 WatchSource:0}: Error finding container 27e07b3b398b41efc61eee6cc12e443c798f41d839d1ef5350894fea09232ec3: Status 404 returned error can't find the container with id 27e07b3b398b41efc61eee6cc12e443c798f41d839d1ef5350894fea09232ec3
Jul 30 18:29:35.103751 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:35.103712    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140152180281904" event={"ID":"98b3c790-eee0-4983-a49f-633122c2605b","Type":"ContainerStarted","Data":"27e07b3b398b41efc61eee6cc12e443c798f41d839d1ef5350894fea09232ec3"}
Jul 30 18:29:35.104921 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:35.104901    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d17a82debd0d9637607d9745a9057612e902ce638dfd10fdf9274f387ec9ab2f"
Jul 30 18:29:35.105029 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:35.104940    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180297264"
Jul 30 18:29:35.115787 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:29:35.115754    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7426-140152180297264\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" podUID="3ad4e9b5-495e-4c92-a0eb-cf6cde03615c" pod="opendatahub/test-curl-7426-140152180297264"
Jul 30 18:29:36.109100 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:29:36.109056    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7426-140152180297264\" is forbidden: User \"system:node:ip-10-0-135-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-203.ec2.internal' and this object" podUID="3ad4e9b5-495e-4c92-a0eb-cf6cde03615c" pod="opendatahub/test-curl-7426-140152180297264"
Jul 30 18:29:36.111460 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:36.111434    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ad4e9b5-495e-4c92-a0eb-cf6cde03615c" path="/var/lib/kubelet/pods/3ad4e9b5-495e-4c92-a0eb-cf6cde03615c/volumes"
Jul 30 18:29:36.112714 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:36.112691    2577 generic.go:356] "Generic (PLEG): container finished" podID="98b3c790-eee0-4983-a49f-633122c2605b" containerID="3795caabde59c927769b7bf637402b0211375567639e3057360e34b0687c9f46" exitCode=0
Jul 30 18:29:36.112853 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:36.112775    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140152180281904" event={"ID":"98b3c790-eee0-4983-a49f-633122c2605b","Type":"ContainerDied","Data":"3795caabde59c927769b7bf637402b0211375567639e3057360e34b0687c9f46"}
Jul 30 18:29:37.266238 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:37.266209    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180281904"
Jul 30 18:29:37.287065 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:37.287035    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7426-140152180281904_98b3c790-eee0-4983-a49f-633122c2605b/test-curl-7426-140152180281904/0.log"
Jul 30 18:29:37.317232 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:37.317195    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7426-140152180281904"]
Jul 30 18:29:37.320785 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:37.320761    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7426-140152180281904"]
Jul 30 18:29:37.389748 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:37.389651    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhhjw\" (UniqueName: \"kubernetes.io/projected/98b3c790-eee0-4983-a49f-633122c2605b-kube-api-access-xhhjw\") pod \"98b3c790-eee0-4983-a49f-633122c2605b\" (UID: \"98b3c790-eee0-4983-a49f-633122c2605b\") "
Jul 30 18:29:37.392004 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:37.391977    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98b3c790-eee0-4983-a49f-633122c2605b-kube-api-access-xhhjw" (OuterVolumeSpecName: "kube-api-access-xhhjw") pod "98b3c790-eee0-4983-a49f-633122c2605b" (UID: "98b3c790-eee0-4983-a49f-633122c2605b"). InnerVolumeSpecName "kube-api-access-xhhjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:29:37.490783 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:37.490750    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhhjw\" (UniqueName: \"kubernetes.io/projected/98b3c790-eee0-4983-a49f-633122c2605b-kube-api-access-xhhjw\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:29:37.503347 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:37.503314    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7426-140152180423824"]
Jul 30 18:29:37.503810 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:37.503796    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98b3c790-eee0-4983-a49f-633122c2605b" containerName="test-curl-7426-140152180281904"
Jul 30 18:29:37.503810 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:37.503826    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="98b3c790-eee0-4983-a49f-633122c2605b" containerName="test-curl-7426-140152180281904"
Jul 30 18:29:37.503946 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:37.503908    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="98b3c790-eee0-4983-a49f-633122c2605b" containerName="test-curl-7426-140152180281904"
Jul 30 18:29:37.506954 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:37.506936    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180423824"
Jul 30 18:29:37.513645 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:37.513616    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140152180423824"]
Jul 30 18:29:37.592052 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:37.592014    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrkvd\" (UniqueName: \"kubernetes.io/projected/cd8328ac-6e9b-4c15-9d06-1968f0f10c7f-kube-api-access-nrkvd\") pod \"test-curl-7426-140152180423824\" (UID: \"cd8328ac-6e9b-4c15-9d06-1968f0f10c7f\") " pod="opendatahub/test-curl-7426-140152180423824"
Jul 30 18:29:37.693222 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:37.693139    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrkvd\" (UniqueName: \"kubernetes.io/projected/cd8328ac-6e9b-4c15-9d06-1968f0f10c7f-kube-api-access-nrkvd\") pod \"test-curl-7426-140152180423824\" (UID: \"cd8328ac-6e9b-4c15-9d06-1968f0f10c7f\") " pod="opendatahub/test-curl-7426-140152180423824"
Jul 30 18:29:37.701408 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:37.701383    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrkvd\" (UniqueName: \"kubernetes.io/projected/cd8328ac-6e9b-4c15-9d06-1968f0f10c7f-kube-api-access-nrkvd\") pod \"test-curl-7426-140152180423824\" (UID: \"cd8328ac-6e9b-4c15-9d06-1968f0f10c7f\") " pod="opendatahub/test-curl-7426-140152180423824"
Jul 30 18:29:37.817869 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:37.817836    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180423824"
Jul 30 18:29:37.956084 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:37.956050    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140152180423824"]
Jul 30 18:29:37.958098 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:29:37.958067    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd8328ac_6e9b_4c15_9d06_1968f0f10c7f.slice/crio-70091f94192d8c50477276ac8e1d7744d52f11d07437f93f6f5e964c2124733e WatchSource:0}: Error finding container 70091f94192d8c50477276ac8e1d7744d52f11d07437f93f6f5e964c2124733e: Status 404 returned error can't find the container with id 70091f94192d8c50477276ac8e1d7744d52f11d07437f93f6f5e964c2124733e
Jul 30 18:29:38.112455 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:38.112422    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98b3c790-eee0-4983-a49f-633122c2605b" path="/var/lib/kubelet/pods/98b3c790-eee0-4983-a49f-633122c2605b/volumes"
Jul 30 18:29:38.125232 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:38.125199    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140152180423824" event={"ID":"cd8328ac-6e9b-4c15-9d06-1968f0f10c7f","Type":"ContainerStarted","Data":"70091f94192d8c50477276ac8e1d7744d52f11d07437f93f6f5e964c2124733e"}
Jul 30 18:29:38.126456 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:38.126438    2577 scope.go:117] "RemoveContainer" containerID="3795caabde59c927769b7bf637402b0211375567639e3057360e34b0687c9f46"
Jul 30 18:29:38.126577 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:38.126442    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180281904"
Jul 30 18:29:39.133115 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:39.133079    2577 generic.go:356] "Generic (PLEG): container finished" podID="cd8328ac-6e9b-4c15-9d06-1968f0f10c7f" containerID="7c45284ca4be2ac7a11a0b55d334fe2ae4eb2e492f7fc760cfa5112139e73e3e" exitCode=0
Jul 30 18:29:39.133568 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:39.133172    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140152180423824" event={"ID":"cd8328ac-6e9b-4c15-9d06-1968f0f10c7f","Type":"ContainerDied","Data":"7c45284ca4be2ac7a11a0b55d334fe2ae4eb2e492f7fc760cfa5112139e73e3e"}
Jul 30 18:29:40.293919 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:40.293887    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180423824"
Jul 30 18:29:40.315609 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:40.315580    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7426-140152180423824_cd8328ac-6e9b-4c15-9d06-1968f0f10c7f/test-curl-7426-140152180423824/0.log"
Jul 30 18:29:40.344134 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:40.344095    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7426-140152180423824"]
Jul 30 18:29:40.350057 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:40.350031    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7426-140152180423824"]
Jul 30 18:29:40.421599 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:40.421504    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrkvd\" (UniqueName: \"kubernetes.io/projected/cd8328ac-6e9b-4c15-9d06-1968f0f10c7f-kube-api-access-nrkvd\") pod \"cd8328ac-6e9b-4c15-9d06-1968f0f10c7f\" (UID: \"cd8328ac-6e9b-4c15-9d06-1968f0f10c7f\") "
Jul 30 18:29:40.423779 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:40.423750    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd8328ac-6e9b-4c15-9d06-1968f0f10c7f-kube-api-access-nrkvd" (OuterVolumeSpecName: "kube-api-access-nrkvd") pod "cd8328ac-6e9b-4c15-9d06-1968f0f10c7f" (UID: "cd8328ac-6e9b-4c15-9d06-1968f0f10c7f"). InnerVolumeSpecName "kube-api-access-nrkvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:29:40.478735 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:40.478704    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7426-140152180423824"]
Jul 30 18:29:40.479178 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:40.479163    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd8328ac-6e9b-4c15-9d06-1968f0f10c7f" containerName="test-curl-7426-140152180423824"
Jul 30 18:29:40.479236 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:40.479180    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd8328ac-6e9b-4c15-9d06-1968f0f10c7f" containerName="test-curl-7426-140152180423824"
Jul 30 18:29:40.479270 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:40.479263    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd8328ac-6e9b-4c15-9d06-1968f0f10c7f" containerName="test-curl-7426-140152180423824"
Jul 30 18:29:40.482583 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:40.482568    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180423824"
Jul 30 18:29:40.487222 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:40.487197    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140152180423824"]
Jul 30 18:29:40.522889 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:40.522860    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrkvd\" (UniqueName: \"kubernetes.io/projected/cd8328ac-6e9b-4c15-9d06-1968f0f10c7f-kube-api-access-nrkvd\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:29:40.623703 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:40.623663    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztjk6\" (UniqueName: \"kubernetes.io/projected/6dfee894-85d1-468d-8dab-609adc2293c7-kube-api-access-ztjk6\") pod \"test-curl-7426-140152180423824\" (UID: \"6dfee894-85d1-468d-8dab-609adc2293c7\") " pod="opendatahub/test-curl-7426-140152180423824"
Jul 30 18:29:40.724841 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:40.724714    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztjk6\" (UniqueName: \"kubernetes.io/projected/6dfee894-85d1-468d-8dab-609adc2293c7-kube-api-access-ztjk6\") pod \"test-curl-7426-140152180423824\" (UID: \"6dfee894-85d1-468d-8dab-609adc2293c7\") " pod="opendatahub/test-curl-7426-140152180423824"
Jul 30 18:29:40.733442 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:40.733409    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztjk6\" (UniqueName: \"kubernetes.io/projected/6dfee894-85d1-468d-8dab-609adc2293c7-kube-api-access-ztjk6\") pod \"test-curl-7426-140152180423824\" (UID: \"6dfee894-85d1-468d-8dab-609adc2293c7\") " pod="opendatahub/test-curl-7426-140152180423824"
Jul 30 18:29:40.794604 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:40.794576    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180423824"
Jul 30 18:29:40.939040 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:40.939011    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140152180423824"]
Jul 30 18:29:40.939707 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:29:40.939686    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6dfee894_85d1_468d_8dab_609adc2293c7.slice/crio-09d2f2fc868d5b61914aabc911b6afc97f2e79fc1674b3d2d05a643e04150fc8 WatchSource:0}: Error finding container 09d2f2fc868d5b61914aabc911b6afc97f2e79fc1674b3d2d05a643e04150fc8: Status 404 returned error can't find the container with id 09d2f2fc868d5b61914aabc911b6afc97f2e79fc1674b3d2d05a643e04150fc8
Jul 30 18:29:41.145969 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:41.145944    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180423824"
Jul 30 18:29:41.145969 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:41.145972    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70091f94192d8c50477276ac8e1d7744d52f11d07437f93f6f5e964c2124733e"
Jul 30 18:29:41.147166 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:41.147140    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140152180423824" event={"ID":"6dfee894-85d1-468d-8dab-609adc2293c7","Type":"ContainerStarted","Data":"09d2f2fc868d5b61914aabc911b6afc97f2e79fc1674b3d2d05a643e04150fc8"}
Jul 30 18:29:41.157529 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:41.157493    2577 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7426-140152180423824" oldPodUID="cd8328ac-6e9b-4c15-9d06-1968f0f10c7f" podUID="6dfee894-85d1-468d-8dab-609adc2293c7"
Jul 30 18:29:42.119738 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:42.119697    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd8328ac-6e9b-4c15-9d06-1968f0f10c7f" path="/var/lib/kubelet/pods/cd8328ac-6e9b-4c15-9d06-1968f0f10c7f/volumes"
Jul 30 18:29:42.154356 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:42.154317    2577 generic.go:356] "Generic (PLEG): container finished" podID="6dfee894-85d1-468d-8dab-609adc2293c7" containerID="ea88e6d19c3419dd43dd3dd0e4eb702cfd0ffed187396e1101e5a947d3594ebe" exitCode=0
Jul 30 18:29:42.154518 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:42.154414    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140152180423824" event={"ID":"6dfee894-85d1-468d-8dab-609adc2293c7","Type":"ContainerDied","Data":"ea88e6d19c3419dd43dd3dd0e4eb702cfd0ffed187396e1101e5a947d3594ebe"}
Jul 30 18:29:43.313779 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:43.313744    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180423824"
Jul 30 18:29:43.336425 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:43.336396    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7426-140152180423824_6dfee894-85d1-468d-8dab-609adc2293c7/test-curl-7426-140152180423824/0.log"
Jul 30 18:29:43.362618 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:43.362579    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7426-140152180423824"]
Jul 30 18:29:43.366671 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:43.366644    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7426-140152180423824"]
Jul 30 18:29:43.450541 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:43.450439    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ztjk6\" (UniqueName: \"kubernetes.io/projected/6dfee894-85d1-468d-8dab-609adc2293c7-kube-api-access-ztjk6\") pod \"6dfee894-85d1-468d-8dab-609adc2293c7\" (UID: \"6dfee894-85d1-468d-8dab-609adc2293c7\") "
Jul 30 18:29:43.452702 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:43.452666    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dfee894-85d1-468d-8dab-609adc2293c7-kube-api-access-ztjk6" (OuterVolumeSpecName: "kube-api-access-ztjk6") pod "6dfee894-85d1-468d-8dab-609adc2293c7" (UID: "6dfee894-85d1-468d-8dab-609adc2293c7"). InnerVolumeSpecName "kube-api-access-ztjk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:29:43.498254 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:43.498220    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7426-140152180256816"]
Jul 30 18:29:43.498686 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:43.498672    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dfee894-85d1-468d-8dab-609adc2293c7" containerName="test-curl-7426-140152180423824"
Jul 30 18:29:43.498735 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:43.498688    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dfee894-85d1-468d-8dab-609adc2293c7" containerName="test-curl-7426-140152180423824"
Jul 30 18:29:43.498772 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:43.498763    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6dfee894-85d1-468d-8dab-609adc2293c7" containerName="test-curl-7426-140152180423824"
Jul 30 18:29:43.501897 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:43.501882    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180256816"
Jul 30 18:29:43.508532 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:43.508504    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140152180256816"]
Jul 30 18:29:43.551339 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:43.551303    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztjk6\" (UniqueName: \"kubernetes.io/projected/6dfee894-85d1-468d-8dab-609adc2293c7-kube-api-access-ztjk6\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:29:43.652372 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:43.652330    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfqbc\" (UniqueName: \"kubernetes.io/projected/84ee39e6-add1-4add-a01d-14fedf2c68bc-kube-api-access-kfqbc\") pod \"test-curl-7426-140152180256816\" (UID: \"84ee39e6-add1-4add-a01d-14fedf2c68bc\") " pod="opendatahub/test-curl-7426-140152180256816"
Jul 30 18:29:43.753667 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:43.753631    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfqbc\" (UniqueName: \"kubernetes.io/projected/84ee39e6-add1-4add-a01d-14fedf2c68bc-kube-api-access-kfqbc\") pod \"test-curl-7426-140152180256816\" (UID: \"84ee39e6-add1-4add-a01d-14fedf2c68bc\") " pod="opendatahub/test-curl-7426-140152180256816"
Jul 30 18:29:43.762680 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:43.762647    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfqbc\" (UniqueName: \"kubernetes.io/projected/84ee39e6-add1-4add-a01d-14fedf2c68bc-kube-api-access-kfqbc\") pod \"test-curl-7426-140152180256816\" (UID: \"84ee39e6-add1-4add-a01d-14fedf2c68bc\") " pod="opendatahub/test-curl-7426-140152180256816"
Jul 30 18:29:43.812510 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:43.812468    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180256816"
Jul 30 18:29:43.949472 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:43.949443    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140152180256816"]
Jul 30 18:29:43.951624 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:29:43.951598    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84ee39e6_add1_4add_a01d_14fedf2c68bc.slice/crio-f0602d651fa77295d09f00adfeffe0715e9cafa8f3b2d47a9d19e7c2c6f4c6d0 WatchSource:0}: Error finding container f0602d651fa77295d09f00adfeffe0715e9cafa8f3b2d47a9d19e7c2c6f4c6d0: Status 404 returned error can't find the container with id f0602d651fa77295d09f00adfeffe0715e9cafa8f3b2d47a9d19e7c2c6f4c6d0
Jul 30 18:29:44.111889 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:44.111765    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6dfee894-85d1-468d-8dab-609adc2293c7" path="/var/lib/kubelet/pods/6dfee894-85d1-468d-8dab-609adc2293c7/volumes"
Jul 30 18:29:44.167896 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:44.167843    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140152180256816" event={"ID":"84ee39e6-add1-4add-a01d-14fedf2c68bc","Type":"ContainerStarted","Data":"f0602d651fa77295d09f00adfeffe0715e9cafa8f3b2d47a9d19e7c2c6f4c6d0"}
Jul 30 18:29:44.169247 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:44.169226    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180423824"
Jul 30 18:29:44.169247 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:44.169242    2577 scope.go:117] "RemoveContainer" containerID="ea88e6d19c3419dd43dd3dd0e4eb702cfd0ffed187396e1101e5a947d3594ebe"
Jul 30 18:29:45.176373 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:45.176341    2577 generic.go:356] "Generic (PLEG): container finished" podID="84ee39e6-add1-4add-a01d-14fedf2c68bc" containerID="fa64fe2c57c15107448e570f67dc841bbc9045ff9af18fcf244b58359a09c04d" exitCode=0
Jul 30 18:29:45.176829 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:45.176428    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140152180256816" event={"ID":"84ee39e6-add1-4add-a01d-14fedf2c68bc","Type":"ContainerDied","Data":"fa64fe2c57c15107448e570f67dc841bbc9045ff9af18fcf244b58359a09c04d"}
Jul 30 18:29:46.322287 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:46.322257    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180256816"
Jul 30 18:29:46.344322 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:46.344295    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7426-140152180256816_84ee39e6-add1-4add-a01d-14fedf2c68bc/test-curl-7426-140152180256816/0.log"
Jul 30 18:29:46.371074 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:46.371046    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7426-140152180256816"]
Jul 30 18:29:46.375854 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:46.375804    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfqbc\" (UniqueName: \"kubernetes.io/projected/84ee39e6-add1-4add-a01d-14fedf2c68bc-kube-api-access-kfqbc\") pod \"84ee39e6-add1-4add-a01d-14fedf2c68bc\" (UID: \"84ee39e6-add1-4add-a01d-14fedf2c68bc\") "
Jul 30 18:29:46.376879 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:46.376854    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7426-140152180256816"]
Jul 30 18:29:46.378069 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:46.378042    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84ee39e6-add1-4add-a01d-14fedf2c68bc-kube-api-access-kfqbc" (OuterVolumeSpecName: "kube-api-access-kfqbc") pod "84ee39e6-add1-4add-a01d-14fedf2c68bc" (UID: "84ee39e6-add1-4add-a01d-14fedf2c68bc"). InnerVolumeSpecName "kube-api-access-kfqbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:29:46.476684 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:46.476602    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfqbc\" (UniqueName: \"kubernetes.io/projected/84ee39e6-add1-4add-a01d-14fedf2c68bc-kube-api-access-kfqbc\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:29:46.512375 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:46.512336    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7426-140152180256816"]
Jul 30 18:29:46.512868 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:46.512851    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84ee39e6-add1-4add-a01d-14fedf2c68bc" containerName="test-curl-7426-140152180256816"
Jul 30 18:29:46.512964 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:46.512870    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="84ee39e6-add1-4add-a01d-14fedf2c68bc" containerName="test-curl-7426-140152180256816"
Jul 30 18:29:46.513035 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:46.512993    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="84ee39e6-add1-4add-a01d-14fedf2c68bc" containerName="test-curl-7426-140152180256816"
Jul 30 18:29:46.516053 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:46.516033    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180256816"
Jul 30 18:29:46.520230 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:46.520205    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140152180256816"]
Jul 30 18:29:46.577968 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:46.577934    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5k4m\" (UniqueName: \"kubernetes.io/projected/f3476254-a9a2-4f59-ab47-ab3ec0c6a810-kube-api-access-p5k4m\") pod \"test-curl-7426-140152180256816\" (UID: \"f3476254-a9a2-4f59-ab47-ab3ec0c6a810\") " pod="opendatahub/test-curl-7426-140152180256816"
Jul 30 18:29:46.679219 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:46.679181    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5k4m\" (UniqueName: \"kubernetes.io/projected/f3476254-a9a2-4f59-ab47-ab3ec0c6a810-kube-api-access-p5k4m\") pod \"test-curl-7426-140152180256816\" (UID: \"f3476254-a9a2-4f59-ab47-ab3ec0c6a810\") " pod="opendatahub/test-curl-7426-140152180256816"
Jul 30 18:29:46.687441 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:46.687406    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5k4m\" (UniqueName: \"kubernetes.io/projected/f3476254-a9a2-4f59-ab47-ab3ec0c6a810-kube-api-access-p5k4m\") pod \"test-curl-7426-140152180256816\" (UID: \"f3476254-a9a2-4f59-ab47-ab3ec0c6a810\") " pod="opendatahub/test-curl-7426-140152180256816"
Jul 30 18:29:46.826786 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:46.826734    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180256816"
Jul 30 18:29:46.966860 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:46.966828    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7426-140152180256816"]
Jul 30 18:29:46.968830 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:29:46.968781    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3476254_a9a2_4f59_ab47_ab3ec0c6a810.slice/crio-7909a093be6412f99af234ce8aefd9ca5d3e6fee8f5cc3f887c8cde937645721 WatchSource:0}: Error finding container 7909a093be6412f99af234ce8aefd9ca5d3e6fee8f5cc3f887c8cde937645721: Status 404 returned error can't find the container with id 7909a093be6412f99af234ce8aefd9ca5d3e6fee8f5cc3f887c8cde937645721
Jul 30 18:29:47.190045 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:47.189959    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180256816"
Jul 30 18:29:47.190045 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:47.189994    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f0602d651fa77295d09f00adfeffe0715e9cafa8f3b2d47a9d19e7c2c6f4c6d0"
Jul 30 18:29:47.191239 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:47.191214    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140152180256816" event={"ID":"f3476254-a9a2-4f59-ab47-ab3ec0c6a810","Type":"ContainerStarted","Data":"7909a093be6412f99af234ce8aefd9ca5d3e6fee8f5cc3f887c8cde937645721"}
Jul 30 18:29:47.201292 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:47.201269    2577 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7426-140152180256816" oldPodUID="84ee39e6-add1-4add-a01d-14fedf2c68bc" podUID="f3476254-a9a2-4f59-ab47-ab3ec0c6a810"
Jul 30 18:29:48.111246 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:48.111209    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84ee39e6-add1-4add-a01d-14fedf2c68bc" path="/var/lib/kubelet/pods/84ee39e6-add1-4add-a01d-14fedf2c68bc/volumes"
Jul 30 18:29:48.198572 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:48.198538    2577 generic.go:356] "Generic (PLEG): container finished" podID="f3476254-a9a2-4f59-ab47-ab3ec0c6a810" containerID="f946efb845aa6d762b1527850adb0a10a00d878a4841f14efea73ba6b6a5a8f2" exitCode=0
Jul 30 18:29:48.198741 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:48.198585    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7426-140152180256816" event={"ID":"f3476254-a9a2-4f59-ab47-ab3ec0c6a810","Type":"ContainerDied","Data":"f946efb845aa6d762b1527850adb0a10a00d878a4841f14efea73ba6b6a5a8f2"}
Jul 30 18:29:49.351432 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:49.351404    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180256816"
Jul 30 18:29:49.372060 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:49.372023    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7426-140152180256816_f3476254-a9a2-4f59-ab47-ab3ec0c6a810/test-curl-7426-140152180256816/0.log"
Jul 30 18:29:49.398598 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:49.398569    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7426-140152180256816"]
Jul 30 18:29:49.401056 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:49.401028    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5k4m\" (UniqueName: \"kubernetes.io/projected/f3476254-a9a2-4f59-ab47-ab3ec0c6a810-kube-api-access-p5k4m\") pod \"f3476254-a9a2-4f59-ab47-ab3ec0c6a810\" (UID: \"f3476254-a9a2-4f59-ab47-ab3ec0c6a810\") "
Jul 30 18:29:49.403041 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:49.403018    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7426-140152180256816"]
Jul 30 18:29:49.403466 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:49.403444    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3476254-a9a2-4f59-ab47-ab3ec0c6a810-kube-api-access-p5k4m" (OuterVolumeSpecName: "kube-api-access-p5k4m") pod "f3476254-a9a2-4f59-ab47-ab3ec0c6a810" (UID: "f3476254-a9a2-4f59-ab47-ab3ec0c6a810"). InnerVolumeSpecName "kube-api-access-p5k4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:29:49.502416 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:49.502379    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5k4m\" (UniqueName: \"kubernetes.io/projected/f3476254-a9a2-4f59-ab47-ab3ec0c6a810-kube-api-access-p5k4m\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:29:49.944478 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:49.944386    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:29:50.112076 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:50.112047    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3476254-a9a2-4f59-ab47-ab3ec0c6a810" path="/var/lib/kubelet/pods/f3476254-a9a2-4f59-ab47-ab3ec0c6a810/volumes"
Jul 30 18:29:50.211136 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:50.211060    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7426-140152180256816"
Jul 30 18:29:50.211136 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:50.211109    2577 scope.go:117] "RemoveContainer" containerID="f946efb845aa6d762b1527850adb0a10a00d878a4841f14efea73ba6b6a5a8f2"
Jul 30 18:29:50.283604 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:50.283553    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:29:50.547165 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:50.547133    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:29:53.443774 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:53.443743    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:29:53.549394 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:53.549360    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:29:54.532747 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:54.532669    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s"]
Jul 30 18:29:54.533389 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:54.533357    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3476254-a9a2-4f59-ab47-ab3ec0c6a810" containerName="test-curl-7426-140152180256816"
Jul 30 18:29:54.533389 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:54.533378    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3476254-a9a2-4f59-ab47-ab3ec0c6a810" containerName="test-curl-7426-140152180256816"
Jul 30 18:29:54.533483 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:54.533473    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3476254-a9a2-4f59-ab47-ab3ec0c6a810" containerName="test-curl-7426-140152180256816"
Jul 30 18:29:54.537535 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:54.537515    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s"
Jul 30 18:29:54.539695 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:54.539671    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-fadb12-serving-cert\""
Jul 30 18:29:54.546636 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:54.546614    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s"]
Jul 30 18:29:54.551500 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:54.551475    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjk8c\" (UniqueName: \"kubernetes.io/projected/659f03bf-a3f4-4c24-9aa0-f436dff0f296-kube-api-access-zjk8c\") pod \"maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s\" (UID: \"659f03bf-a3f4-4c24-9aa0-f436dff0f296\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s"
Jul 30 18:29:54.551606 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:54.551542    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/659f03bf-a3f4-4c24-9aa0-f436dff0f296-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s\" (UID: \"659f03bf-a3f4-4c24-9aa0-f436dff0f296\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s"
Jul 30 18:29:54.551687 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:54.551622    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/659f03bf-a3f4-4c24-9aa0-f436dff0f296-maas-api-tls\") pod \"maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s\" (UID: \"659f03bf-a3f4-4c24-9aa0-f436dff0f296\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s"
Jul 30 18:29:54.565285 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:54.565263    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:29:54.652590 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:54.652551    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjk8c\" (UniqueName: \"kubernetes.io/projected/659f03bf-a3f4-4c24-9aa0-f436dff0f296-kube-api-access-zjk8c\") pod \"maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s\" (UID: \"659f03bf-a3f4-4c24-9aa0-f436dff0f296\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s"
Jul 30 18:29:54.652762 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:54.652606    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/659f03bf-a3f4-4c24-9aa0-f436dff0f296-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s\" (UID: \"659f03bf-a3f4-4c24-9aa0-f436dff0f296\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s"
Jul 30 18:29:54.652762 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:54.652644    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/659f03bf-a3f4-4c24-9aa0-f436dff0f296-maas-api-tls\") pod \"maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s\" (UID: \"659f03bf-a3f4-4c24-9aa0-f436dff0f296\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s"
Jul 30 18:29:54.652909 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:29:54.652771    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-serving-cert: secret "maas-api-e2e-ipp-a-fadb12-serving-cert" not found
Jul 30 18:29:54.652909 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:29:54.652854    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/659f03bf-a3f4-4c24-9aa0-f436dff0f296-maas-api-tls podName:659f03bf-a3f4-4c24-9aa0-f436dff0f296 nodeName:}" failed. No retries permitted until 2026-07-30 18:29:55.152837925 +0000 UTC m=+3769.606401894 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/659f03bf-a3f4-4c24-9aa0-f436dff0f296-maas-api-tls") pod "maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s" (UID: "659f03bf-a3f4-4c24-9aa0-f436dff0f296") : secret "maas-api-e2e-ipp-a-fadb12-serving-cert" not found
Jul 30 18:29:54.653306 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:54.653281    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/659f03bf-a3f4-4c24-9aa0-f436dff0f296-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s\" (UID: \"659f03bf-a3f4-4c24-9aa0-f436dff0f296\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s"
Jul 30 18:29:54.662366 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:54.662342    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjk8c\" (UniqueName: \"kubernetes.io/projected/659f03bf-a3f4-4c24-9aa0-f436dff0f296-kube-api-access-zjk8c\") pod \"maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s\" (UID: \"659f03bf-a3f4-4c24-9aa0-f436dff0f296\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s"
Jul 30 18:29:54.791234 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:54.791135    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:29:55.080663 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.080569    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:29:55.159457 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.159408    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/659f03bf-a3f4-4c24-9aa0-f436dff0f296-maas-api-tls\") pod \"maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s\" (UID: \"659f03bf-a3f4-4c24-9aa0-f436dff0f296\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s"
Jul 30 18:29:55.162159 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.162131    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/659f03bf-a3f4-4c24-9aa0-f436dff0f296-maas-api-tls\") pod \"maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s\" (UID: \"659f03bf-a3f4-4c24-9aa0-f436dff0f296\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s"
Jul 30 18:29:55.179281 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.179252    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4"]
Jul 30 18:29:55.182421 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.182396    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:29:55.182930 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.182908    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4"
Jul 30 18:29:55.185096 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.185074    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-fadb12\""
Jul 30 18:29:55.194923 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.194890    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4"]
Jul 30 18:29:55.214358 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.214331    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk"]
Jul 30 18:29:55.217745 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.217730    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk"
Jul 30 18:29:55.220547 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.220527    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-fadb12-dockercfg-tstkx\""
Jul 30 18:29:55.230136 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.230112    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk"]
Jul 30 18:29:55.260693 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.260664    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6f64\" (UniqueName: \"kubernetes.io/projected/53c3ea19-33b3-4644-8b15-0daafaedad18-kube-api-access-x6f64\") pod \"payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4\" (UID: \"53c3ea19-33b3-4644-8b15-0daafaedad18\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4"
Jul 30 18:29:55.260853 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.260711    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53c3ea19-33b3-4644-8b15-0daafaedad18-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4\" (UID: \"53c3ea19-33b3-4644-8b15-0daafaedad18\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4"
Jul 30 18:29:55.260957 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.260932    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjtxn\" (UniqueName: \"kubernetes.io/projected/f7d84819-6c7e-4b3b-8a16-97b9d99c6609-kube-api-access-hjtxn\") pod \"payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk\" (UID: \"f7d84819-6c7e-4b3b-8a16-97b9d99c6609\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk"
Jul 30 18:29:55.261036 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.261021    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7d84819-6c7e-4b3b-8a16-97b9d99c6609-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk\" (UID: \"f7d84819-6c7e-4b3b-8a16-97b9d99c6609\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk"
Jul 30 18:29:55.362061 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.361951    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7d84819-6c7e-4b3b-8a16-97b9d99c6609-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk\" (UID: \"f7d84819-6c7e-4b3b-8a16-97b9d99c6609\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk"
Jul 30 18:29:55.362061 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.362017    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6f64\" (UniqueName: \"kubernetes.io/projected/53c3ea19-33b3-4644-8b15-0daafaedad18-kube-api-access-x6f64\") pod \"payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4\" (UID: \"53c3ea19-33b3-4644-8b15-0daafaedad18\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4"
Jul 30 18:29:55.362061 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.362058    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53c3ea19-33b3-4644-8b15-0daafaedad18-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4\" (UID: \"53c3ea19-33b3-4644-8b15-0daafaedad18\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4"
Jul 30 18:29:55.362294 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.362187    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjtxn\" (UniqueName: \"kubernetes.io/projected/f7d84819-6c7e-4b3b-8a16-97b9d99c6609-kube-api-access-hjtxn\") pod \"payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk\" (UID: \"f7d84819-6c7e-4b3b-8a16-97b9d99c6609\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk"
Jul 30 18:29:55.362972 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.362946    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7d84819-6c7e-4b3b-8a16-97b9d99c6609-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk\" (UID: \"f7d84819-6c7e-4b3b-8a16-97b9d99c6609\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk"
Jul 30 18:29:55.363427 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.363406    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53c3ea19-33b3-4644-8b15-0daafaedad18-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4\" (UID: \"53c3ea19-33b3-4644-8b15-0daafaedad18\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4"
Jul 30 18:29:55.370197 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.370168    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjtxn\" (UniqueName: \"kubernetes.io/projected/f7d84819-6c7e-4b3b-8a16-97b9d99c6609-kube-api-access-hjtxn\") pod \"payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk\" (UID: \"f7d84819-6c7e-4b3b-8a16-97b9d99c6609\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk"
Jul 30 18:29:55.370316 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.370268    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6f64\" (UniqueName: \"kubernetes.io/projected/53c3ea19-33b3-4644-8b15-0daafaedad18-kube-api-access-x6f64\") pod \"payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4\" (UID: \"53c3ea19-33b3-4644-8b15-0daafaedad18\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4"
Jul 30 18:29:55.448643 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.448613    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s"
Jul 30 18:29:55.493806 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.493675    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4"
Jul 30 18:29:55.529881 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.529847    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk"
Jul 30 18:29:55.617518 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.617007    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s"]
Jul 30 18:29:55.620011 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:29:55.619522    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod659f03bf_a3f4_4c24_9aa0_f436dff0f296.slice/crio-4cac60764941db6693f9b59d605302693d80363e06ef4a2250a454a6b3717abb WatchSource:0}: Error finding container 4cac60764941db6693f9b59d605302693d80363e06ef4a2250a454a6b3717abb: Status 404 returned error can't find the container with id 4cac60764941db6693f9b59d605302693d80363e06ef4a2250a454a6b3717abb
Jul 30 18:29:55.720506 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.720475    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk"]
Jul 30 18:29:55.723326 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:29:55.723293    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7d84819_6c7e_4b3b_8a16_97b9d99c6609.slice/crio-56e40b84931298c8a3040da9468d1f42f7f59a2a3745eb51d7a6979390668df4 WatchSource:0}: Error finding container 56e40b84931298c8a3040da9468d1f42f7f59a2a3745eb51d7a6979390668df4: Status 404 returned error can't find the container with id 56e40b84931298c8a3040da9468d1f42f7f59a2a3745eb51d7a6979390668df4
Jul 30 18:29:55.875306 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:55.875176    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4"]
Jul 30 18:29:55.878613 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:29:55.878578    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53c3ea19_33b3_4644_8b15_0daafaedad18.slice/crio-8521b7b8aeec8c2ab5cc500f8d3c181fe9e896224595efe772daacb696b81a12 WatchSource:0}: Error finding container 8521b7b8aeec8c2ab5cc500f8d3c181fe9e896224595efe772daacb696b81a12: Status 404 returned error can't find the container with id 8521b7b8aeec8c2ab5cc500f8d3c181fe9e896224595efe772daacb696b81a12
Jul 30 18:29:56.251192 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:56.251158    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4" event={"ID":"53c3ea19-33b3-4644-8b15-0daafaedad18","Type":"ContainerStarted","Data":"a2ac89b5c9045c3a1502a656b84219eafdac5d7331e7302d6576798c7f9f21c1"}
Jul 30 18:29:56.251401 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:56.251220    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4" event={"ID":"53c3ea19-33b3-4644-8b15-0daafaedad18","Type":"ContainerStarted","Data":"8521b7b8aeec8c2ab5cc500f8d3c181fe9e896224595efe772daacb696b81a12"}
Jul 30 18:29:56.251401 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:56.251364    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4"
Jul 30 18:29:56.252837 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:56.252796    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s" event={"ID":"659f03bf-a3f4-4c24-9aa0-f436dff0f296","Type":"ContainerStarted","Data":"301bde8adf98be1f26947c67c9ffc39b304b24e8ae73a65c912c7d4e188413d4"}
Jul 30 18:29:56.252974 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:56.252861    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s" event={"ID":"659f03bf-a3f4-4c24-9aa0-f436dff0f296","Type":"ContainerStarted","Data":"4cac60764941db6693f9b59d605302693d80363e06ef4a2250a454a6b3717abb"}
Jul 30 18:29:56.252974 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:56.252889    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s"
Jul 30 18:29:56.254503 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:56.254478    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk" event={"ID":"f7d84819-6c7e-4b3b-8a16-97b9d99c6609","Type":"ContainerStarted","Data":"6391fd6df9c8ed0e7565a0b5917320dd7d43aec858a29398f1525538a4a58830"}
Jul 30 18:29:56.254623 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:56.254525    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk" event={"ID":"f7d84819-6c7e-4b3b-8a16-97b9d99c6609","Type":"ContainerStarted","Data":"56e40b84931298c8a3040da9468d1f42f7f59a2a3745eb51d7a6979390668df4"}
Jul 30 18:29:56.254623 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:56.254552    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk"
Jul 30 18:29:56.268331 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:56.268259    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4" podStartSLOduration=1.26823544 podStartE2EDuration="1.26823544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:29:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:29:56.266180196 +0000 UTC m=+3770.719744189" watchObservedRunningTime="2026-07-30 18:29:56.26823544 +0000 UTC m=+3770.721799485"
Jul 30 18:29:56.283690 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:56.283631    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s" podStartSLOduration=1.772487876 podStartE2EDuration="2.28361368s" totalImagesPullingTime="511.125804ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:29:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:29:56.282758876 +0000 UTC m=+3770.736322868" watchObservedRunningTime="2026-07-30 18:29:56.28361368 +0000 UTC m=+3770.737177670"
Jul 30 18:29:56.299063 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:29:56.299013    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk" podStartSLOduration=1.298996837 podStartE2EDuration="1.298996837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:29:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:29:56.297335974 +0000 UTC m=+3770.750899995" watchObservedRunningTime="2026-07-30 18:29:56.298996837 +0000 UTC m=+3770.752560893"
Jul 30 18:30:02.268716 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:02.268684    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s"
Jul 30 18:30:04.114744 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:04.114704    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757225-7lrcp"]
Jul 30 18:30:04.122666 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:04.122632    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757225-7lrcp"]
Jul 30 18:30:06.111889 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:06.111856    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="940576c4-8056-4abe-99d1-253cefffb3c3" path="/var/lib/kubelet/pods/940576c4-8056-4abe-99d1-253cefffb3c3/volumes"
Jul 30 18:30:06.593807 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:06.593773    2577 scope.go:117] "RemoveContainer" containerID="0cafbb5b549bc24f6dd31f5855cea601119fda35f15ba428737eb6e8e73a8a57"
Jul 30 18:30:07.266042 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:07.266015    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4"
Jul 30 18:30:07.266483 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:07.266288    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk"
Jul 30 18:30:18.503553 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:18.503489    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:30:19.229321 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:19.229263    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:30:19.558004 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:19.557965    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:30:19.661467 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:19.661424    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:30:24.973779 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:24.973737    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:30:25.074026 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:25.073991    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:30:26.130563 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:26.130531    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8"]
Jul 30 18:30:26.133550 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:26.133530    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8"
Jul 30 18:30:26.136107 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:26.136090    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-c8d7de-serving-cert\""
Jul 30 18:30:26.146477 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:26.146451    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8"]
Jul 30 18:30:26.176834 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:26.176783    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8\" (UID: \"fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8"
Jul 30 18:30:26.177022 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:26.176872    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8\" (UID: \"fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8"
Jul 30 18:30:26.177101 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:26.177069    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4qdr\" (UniqueName: \"kubernetes.io/projected/fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856-kube-api-access-f4qdr\") pod \"maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8\" (UID: \"fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8"
Jul 30 18:30:26.227638 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:26.227582    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:30:26.277871 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:26.277808    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8\" (UID: \"fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8"
Jul 30 18:30:26.278064 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:26.277892    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8\" (UID: \"fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8"
Jul 30 18:30:26.278064 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:26.277949    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4qdr\" (UniqueName: \"kubernetes.io/projected/fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856-kube-api-access-f4qdr\") pod \"maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8\" (UID: \"fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8"
Jul 30 18:30:26.278064 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:30:26.277953    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-serving-cert: secret "maas-api-e2e-ipp-b-c8d7de-serving-cert" not found
Jul 30 18:30:26.278064 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:30:26.278022    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856-maas-api-tls podName:fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856 nodeName:}" failed. No retries permitted until 2026-07-30 18:30:26.778005045 +0000 UTC m=+3801.231569013 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856-maas-api-tls") pod "maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8" (UID: "fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856") : secret "maas-api-e2e-ipp-b-c8d7de-serving-cert" not found
Jul 30 18:30:26.278586 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:26.278556    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8\" (UID: \"fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8"
Jul 30 18:30:26.290870 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:26.290804    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4qdr\" (UniqueName: \"kubernetes.io/projected/fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856-kube-api-access-f4qdr\") pod \"maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8\" (UID: \"fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8"
Jul 30 18:30:26.354530 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:26.354498    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:30:26.782133 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:26.782098    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8\" (UID: \"fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8"
Jul 30 18:30:26.784781 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:26.784757    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8\" (UID: \"fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8"
Jul 30 18:30:26.857740 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:26.857704    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:30:26.961903 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:26.961863    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:30:26.981493 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:26.981447    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc"]
Jul 30 18:30:26.986090 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:26.986055    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc"
Jul 30 18:30:26.988451 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:26.988424    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-c8d7de\""
Jul 30 18:30:26.988727 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:26.988702    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-c8d7de-dockercfg-8rxlz\""
Jul 30 18:30:26.994688 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:26.994572    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc"]
Jul 30 18:30:27.043509 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:27.043423    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8"
Jul 30 18:30:27.084572 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:27.084524    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07ca1695-a48d-4d84-84aa-6839a06d3066-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc\" (UID: \"07ca1695-a48d-4d84-84aa-6839a06d3066\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc"
Jul 30 18:30:27.084773 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:27.084594    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45qvz\" (UniqueName: \"kubernetes.io/projected/07ca1695-a48d-4d84-84aa-6839a06d3066-kube-api-access-45qvz\") pod \"payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc\" (UID: \"07ca1695-a48d-4d84-84aa-6839a06d3066\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc"
Jul 30 18:30:27.185890 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:27.185856    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45qvz\" (UniqueName: \"kubernetes.io/projected/07ca1695-a48d-4d84-84aa-6839a06d3066-kube-api-access-45qvz\") pod \"payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc\" (UID: \"07ca1695-a48d-4d84-84aa-6839a06d3066\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc"
Jul 30 18:30:27.186322 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:27.186186    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07ca1695-a48d-4d84-84aa-6839a06d3066-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc\" (UID: \"07ca1695-a48d-4d84-84aa-6839a06d3066\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc"
Jul 30 18:30:27.186870 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:27.186848    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07ca1695-a48d-4d84-84aa-6839a06d3066-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc\" (UID: \"07ca1695-a48d-4d84-84aa-6839a06d3066\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc"
Jul 30 18:30:27.198333 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:27.198310    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8"]
Jul 30 18:30:27.198581 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:27.198565    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45qvz\" (UniqueName: \"kubernetes.io/projected/07ca1695-a48d-4d84-84aa-6839a06d3066-kube-api-access-45qvz\") pod \"payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc\" (UID: \"07ca1695-a48d-4d84-84aa-6839a06d3066\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc"
Jul 30 18:30:27.200489 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:30:27.200462    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcc9bd89_5e68_4ed5_b6ad_ef7dde6b6856.slice/crio-b15d20a3d8374d771db12238d6d1e5eb026731abda671c7381f2dbb108315221 WatchSource:0}: Error finding container b15d20a3d8374d771db12238d6d1e5eb026731abda671c7381f2dbb108315221: Status 404 returned error can't find the container with id b15d20a3d8374d771db12238d6d1e5eb026731abda671c7381f2dbb108315221
Jul 30 18:30:27.300293 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:27.300207    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc"
Jul 30 18:30:27.441282 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:27.441244    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8" event={"ID":"fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856","Type":"ContainerStarted","Data":"b15d20a3d8374d771db12238d6d1e5eb026731abda671c7381f2dbb108315221"}
Jul 30 18:30:27.446304 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:27.446281    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc"]
Jul 30 18:30:27.448336 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:30:27.448308    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07ca1695_a48d_4d84_84aa_6839a06d3066.slice/crio-2a20ba68fa13f98059ce6dc0e6e3dec3671e33fc24e6976a86b260a753c080ce WatchSource:0}: Error finding container 2a20ba68fa13f98059ce6dc0e6e3dec3671e33fc24e6976a86b260a753c080ce: Status 404 returned error can't find the container with id 2a20ba68fa13f98059ce6dc0e6e3dec3671e33fc24e6976a86b260a753c080ce
Jul 30 18:30:28.448617 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:28.448585    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc" event={"ID":"07ca1695-a48d-4d84-84aa-6839a06d3066","Type":"ContainerStarted","Data":"d1cca186a602179ba880a91b585e1800ac6046057756689b0277b9da0e642e23"}
Jul 30 18:30:28.449150 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:28.448644    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc" event={"ID":"07ca1695-a48d-4d84-84aa-6839a06d3066","Type":"ContainerStarted","Data":"2a20ba68fa13f98059ce6dc0e6e3dec3671e33fc24e6976a86b260a753c080ce"}
Jul 30 18:30:28.449150 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:28.448772    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc"
Jul 30 18:30:28.450304 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:28.450284    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8" event={"ID":"fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856","Type":"ContainerStarted","Data":"084e57df0e215aca592ed7dfaffa90510fb3a7dcbe1494e5ea586fc49fdc5bd6"}
Jul 30 18:30:28.450409 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:28.450397    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8"
Jul 30 18:30:28.467484 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:28.467436    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc" podStartSLOduration=2.467422544 podStartE2EDuration="2.467422544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:30:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:30:28.464744907 +0000 UTC m=+3802.918308902" watchObservedRunningTime="2026-07-30 18:30:28.467422544 +0000 UTC m=+3802.920986535"
Jul 30 18:30:28.484449 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:28.484402    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8" podStartSLOduration=2.197322295 podStartE2EDuration="2.484382692s" totalImagesPullingTime="287.060397ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:30:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:30:28.481259554 +0000 UTC m=+3802.934823547" watchObservedRunningTime="2026-07-30 18:30:28.484382692 +0000 UTC m=+3802.937946683"
Jul 30 18:30:34.463178 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:34.463149    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8"
Jul 30 18:30:39.457628 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:39.457540    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc"
Jul 30 18:30:50.035189 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:50.035153    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:30:57.154106 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:57.154075    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-t25kk_335be1a6-121e-41a2-b390-180e8c87ccdf/payload-processing/0.log"
Jul 30 18:30:57.286781 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:57.286754    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc_07ca1695-a48d-4d84-84aa-6839a06d3066/payload-processing/0.log"
Jul 30 18:30:59.944040 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:30:59.944009    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:31:47.378834 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:47.378789    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk_f7d84819-6c7e-4b3b-8a16-97b9d99c6609/payload-processing/0.log"
Jul 30 18:31:47.506645 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:47.506612    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-t25kk_335be1a6-121e-41a2-b390-180e8c87ccdf/payload-processing/0.log"
Jul 30 18:31:47.639748 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:47.639670    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc_07ca1695-a48d-4d84-84aa-6839a06d3066/payload-processing/0.log"
Jul 30 18:31:47.940267 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:47.940184    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:31:48.266168 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:48.266111    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:31:48.562873 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:48.562768    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:31:51.424984 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:51.424927    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:31:51.523379 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:51.523324    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:31:52.525348 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:52.525313    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz"]
Jul 30 18:31:52.528565 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:52.528543    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz"
Jul 30 18:31:52.530683 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:52.530666    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-46b4e2-serving-cert\""
Jul 30 18:31:52.538772 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:52.538747    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz"]
Jul 30 18:31:52.625044 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:52.625009    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:31:52.669978 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:52.669951    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcdf4e0d-1915-44b2-ad00-374338dd1e90-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz\" (UID: \"dcdf4e0d-1915-44b2-ad00-374338dd1e90\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz"
Jul 30 18:31:52.670154 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:52.669990    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc4jg\" (UniqueName: \"kubernetes.io/projected/dcdf4e0d-1915-44b2-ad00-374338dd1e90-kube-api-access-kc4jg\") pod \"maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz\" (UID: \"dcdf4e0d-1915-44b2-ad00-374338dd1e90\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz"
Jul 30 18:31:52.670154 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:52.670082    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcdf4e0d-1915-44b2-ad00-374338dd1e90-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz\" (UID: \"dcdf4e0d-1915-44b2-ad00-374338dd1e90\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz"
Jul 30 18:31:52.750562 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:52.750518    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:31:52.770793 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:52.770764    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcdf4e0d-1915-44b2-ad00-374338dd1e90-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz\" (UID: \"dcdf4e0d-1915-44b2-ad00-374338dd1e90\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz"
Jul 30 18:31:52.770995 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:52.770886    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcdf4e0d-1915-44b2-ad00-374338dd1e90-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz\" (UID: \"dcdf4e0d-1915-44b2-ad00-374338dd1e90\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz"
Jul 30 18:31:52.770995 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:52.770923    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc4jg\" (UniqueName: \"kubernetes.io/projected/dcdf4e0d-1915-44b2-ad00-374338dd1e90-kube-api-access-kc4jg\") pod \"maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz\" (UID: \"dcdf4e0d-1915-44b2-ad00-374338dd1e90\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz"
Jul 30 18:31:52.771504 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:52.771477    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcdf4e0d-1915-44b2-ad00-374338dd1e90-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz\" (UID: \"dcdf4e0d-1915-44b2-ad00-374338dd1e90\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz"
Jul 30 18:31:52.773453 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:52.773433    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcdf4e0d-1915-44b2-ad00-374338dd1e90-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz\" (UID: \"dcdf4e0d-1915-44b2-ad00-374338dd1e90\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz"
Jul 30 18:31:52.779977 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:52.779922    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc4jg\" (UniqueName: \"kubernetes.io/projected/dcdf4e0d-1915-44b2-ad00-374338dd1e90-kube-api-access-kc4jg\") pod \"maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz\" (UID: \"dcdf4e0d-1915-44b2-ad00-374338dd1e90\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz"
Jul 30 18:31:52.840336 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:52.840302    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz"
Jul 30 18:31:52.993085 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:52.993061    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz"]
Jul 30 18:31:52.995016 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:31:52.994980    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcdf4e0d_1915_44b2_ad00_374338dd1e90.slice/crio-81cc6b6e54b06ef0651014b1eed03c1b1402db1467b45e512c53bedaae6f03a6 WatchSource:0}: Error finding container 81cc6b6e54b06ef0651014b1eed03c1b1402db1467b45e512c53bedaae6f03a6: Status 404 returned error can't find the container with id 81cc6b6e54b06ef0651014b1eed03c1b1402db1467b45e512c53bedaae6f03a6
Jul 30 18:31:53.114981 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:53.114883    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:31:53.216449 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:53.216416    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:31:53.940504 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:53.940471    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz" event={"ID":"dcdf4e0d-1915-44b2-ad00-374338dd1e90","Type":"ContainerStarted","Data":"c6367346ee3d33d4839f144b12d05c0a6e64930f86f90448c45bb2c43d29e914"}
Jul 30 18:31:53.940898 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:53.940520    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz" event={"ID":"dcdf4e0d-1915-44b2-ad00-374338dd1e90","Type":"ContainerStarted","Data":"81cc6b6e54b06ef0651014b1eed03c1b1402db1467b45e512c53bedaae6f03a6"}
Jul 30 18:31:53.940898 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:53.940673    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz"
Jul 30 18:31:53.959188 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:53.959124    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz" podStartSLOduration=1.122572261 podStartE2EDuration="1.959103036s" totalImagesPullingTime="836.530775ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:31:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:31:53.957961703 +0000 UTC m=+3888.411525695" watchObservedRunningTime="2026-07-30 18:31:53.959103036 +0000 UTC m=+3888.412667028"
Jul 30 18:31:54.243256 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:54.243165    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:31:54.304756 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:54.304723    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:31:59.950969 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:31:59.950937    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz"
Jul 30 18:32:16.511713 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:16.511676    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:32:23.299348 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:23.299313    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz"]
Jul 30 18:32:23.299801 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:23.299631    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz" podUID="dcdf4e0d-1915-44b2-ad00-374338dd1e90" containerName="maas-api" containerID="cri-o://c6367346ee3d33d4839f144b12d05c0a6e64930f86f90448c45bb2c43d29e914" gracePeriod=30
Jul 30 18:32:23.332131 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:23.332106    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:32:23.407672 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:23.407444    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:32:23.470160 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:23.470110    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:32:23.532412 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:23.532383    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:32:23.561917 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:23.561853    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz"
Jul 30 18:32:23.677055 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:23.677022    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcdf4e0d-1915-44b2-ad00-374338dd1e90-openshift-service-ca\") pod \"dcdf4e0d-1915-44b2-ad00-374338dd1e90\" (UID: \"dcdf4e0d-1915-44b2-ad00-374338dd1e90\") "
Jul 30 18:32:23.677329 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:23.677069    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcdf4e0d-1915-44b2-ad00-374338dd1e90-maas-api-tls\") pod \"dcdf4e0d-1915-44b2-ad00-374338dd1e90\" (UID: \"dcdf4e0d-1915-44b2-ad00-374338dd1e90\") "
Jul 30 18:32:23.677329 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:23.677142    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kc4jg\" (UniqueName: \"kubernetes.io/projected/dcdf4e0d-1915-44b2-ad00-374338dd1e90-kube-api-access-kc4jg\") pod \"dcdf4e0d-1915-44b2-ad00-374338dd1e90\" (UID: \"dcdf4e0d-1915-44b2-ad00-374338dd1e90\") "
Jul 30 18:32:23.677472 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:23.677443    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcdf4e0d-1915-44b2-ad00-374338dd1e90-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dcdf4e0d-1915-44b2-ad00-374338dd1e90" (UID: "dcdf4e0d-1915-44b2-ad00-374338dd1e90"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:32:23.679278 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:23.679259    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcdf4e0d-1915-44b2-ad00-374338dd1e90-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dcdf4e0d-1915-44b2-ad00-374338dd1e90" (UID: "dcdf4e0d-1915-44b2-ad00-374338dd1e90"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 18:32:23.679357 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:23.679334    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcdf4e0d-1915-44b2-ad00-374338dd1e90-kube-api-access-kc4jg" (OuterVolumeSpecName: "kube-api-access-kc4jg") pod "dcdf4e0d-1915-44b2-ad00-374338dd1e90" (UID: "dcdf4e0d-1915-44b2-ad00-374338dd1e90"). InnerVolumeSpecName "kube-api-access-kc4jg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:32:23.778465 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:23.778431    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kc4jg\" (UniqueName: \"kubernetes.io/projected/dcdf4e0d-1915-44b2-ad00-374338dd1e90-kube-api-access-kc4jg\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:32:23.778465 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:23.778465    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcdf4e0d-1915-44b2-ad00-374338dd1e90-openshift-service-ca\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:32:23.778465 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:23.778475    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcdf4e0d-1915-44b2-ad00-374338dd1e90-maas-api-tls\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:32:23.918030 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:23.917932    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:32:24.115324 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:24.115287    2577 generic.go:356] "Generic (PLEG): container finished" podID="dcdf4e0d-1915-44b2-ad00-374338dd1e90" containerID="c6367346ee3d33d4839f144b12d05c0a6e64930f86f90448c45bb2c43d29e914" exitCode=0
Jul 30 18:32:24.115489 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:24.115335    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz"
Jul 30 18:32:24.115489 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:24.115380    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz" event={"ID":"dcdf4e0d-1915-44b2-ad00-374338dd1e90","Type":"ContainerDied","Data":"c6367346ee3d33d4839f144b12d05c0a6e64930f86f90448c45bb2c43d29e914"}
Jul 30 18:32:24.115489 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:24.115439    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz" event={"ID":"dcdf4e0d-1915-44b2-ad00-374338dd1e90","Type":"ContainerDied","Data":"81cc6b6e54b06ef0651014b1eed03c1b1402db1467b45e512c53bedaae6f03a6"}
Jul 30 18:32:24.115489 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:24.115474    2577 scope.go:117] "RemoveContainer" containerID="c6367346ee3d33d4839f144b12d05c0a6e64930f86f90448c45bb2c43d29e914"
Jul 30 18:32:24.125949 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:24.125927    2577 scope.go:117] "RemoveContainer" containerID="c6367346ee3d33d4839f144b12d05c0a6e64930f86f90448c45bb2c43d29e914"
Jul 30 18:32:24.126210 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:32:24.126193    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6367346ee3d33d4839f144b12d05c0a6e64930f86f90448c45bb2c43d29e914\": container with ID starting with c6367346ee3d33d4839f144b12d05c0a6e64930f86f90448c45bb2c43d29e914 not found: ID does not exist" containerID="c6367346ee3d33d4839f144b12d05c0a6e64930f86f90448c45bb2c43d29e914"
Jul 30 18:32:24.126292 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:24.126222    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6367346ee3d33d4839f144b12d05c0a6e64930f86f90448c45bb2c43d29e914"} err="failed to get container status \"c6367346ee3d33d4839f144b12d05c0a6e64930f86f90448c45bb2c43d29e914\": rpc error: code = NotFound desc = could not find container \"c6367346ee3d33d4839f144b12d05c0a6e64930f86f90448c45bb2c43d29e914\": container with ID starting with c6367346ee3d33d4839f144b12d05c0a6e64930f86f90448c45bb2c43d29e914 not found: ID does not exist"
Jul 30 18:32:24.136100 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:24.136079    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz"]
Jul 30 18:32:24.140044 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:24.140009    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-46b4e2-58576f55c7-zdghz"]
Jul 30 18:32:26.111390 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:26.111353    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcdf4e0d-1915-44b2-ad00-374338dd1e90" path="/var/lib/kubelet/pods/dcdf4e0d-1915-44b2-ad00-374338dd1e90/volumes"
Jul 30 18:32:29.946060 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:29.946028    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:32:30.095035 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:30.095008    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:32:30.296781 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:30.296755    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:32:30.526708 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:30.526678    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:32:33.895367 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:33.895338    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:32:33.995488 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:33.995458    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:32:49.752476 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:49.752427    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s"]
Jul 30 18:32:49.752995 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:49.752642    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s" podUID="659f03bf-a3f4-4c24-9aa0-f436dff0f296" containerName="maas-api" containerID="cri-o://301bde8adf98be1f26947c67c9ffc39b304b24e8ae73a65c912c7d4e188413d4" gracePeriod=30
Jul 30 18:32:49.792928 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:49.792898    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:32:49.826245 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:49.826211    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk"]
Jul 30 18:32:49.826508 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:49.826476    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk" podUID="f7d84819-6c7e-4b3b-8a16-97b9d99c6609" containerName="payload-processing" containerID="cri-o://6391fd6df9c8ed0e7565a0b5917320dd7d43aec858a29398f1525538a4a58830" gracePeriod=30
Jul 30 18:32:49.832029 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:49.832002    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4"]
Jul 30 18:32:49.832243 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:49.832222    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4" podUID="53c3ea19-33b3-4644-8b15-0daafaedad18" containerName="payload-pre-processing" containerID="cri-o://a2ac89b5c9045c3a1502a656b84219eafdac5d7331e7302d6576798c7f9f21c1" gracePeriod=30
Jul 30 18:32:49.863757 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:49.863676    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:32:49.926091 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:49.926051    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:32:50.026756 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.026628    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:32:50.049304 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.049274    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s"
Jul 30 18:32:50.118749 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.118715    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk"
Jul 30 18:32:50.129132 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.129107    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4"
Jul 30 18:32:50.153563 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.153539    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/659f03bf-a3f4-4c24-9aa0-f436dff0f296-openshift-service-ca\") pod \"659f03bf-a3f4-4c24-9aa0-f436dff0f296\" (UID: \"659f03bf-a3f4-4c24-9aa0-f436dff0f296\") "
Jul 30 18:32:50.153732 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.153678    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjk8c\" (UniqueName: \"kubernetes.io/projected/659f03bf-a3f4-4c24-9aa0-f436dff0f296-kube-api-access-zjk8c\") pod \"659f03bf-a3f4-4c24-9aa0-f436dff0f296\" (UID: \"659f03bf-a3f4-4c24-9aa0-f436dff0f296\") "
Jul 30 18:32:50.153798 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.153750    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/659f03bf-a3f4-4c24-9aa0-f436dff0f296-maas-api-tls\") pod \"659f03bf-a3f4-4c24-9aa0-f436dff0f296\" (UID: \"659f03bf-a3f4-4c24-9aa0-f436dff0f296\") "
Jul 30 18:32:50.153949 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.153923    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/659f03bf-a3f4-4c24-9aa0-f436dff0f296-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "659f03bf-a3f4-4c24-9aa0-f436dff0f296" (UID: "659f03bf-a3f4-4c24-9aa0-f436dff0f296"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:32:50.154200 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.154180    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/659f03bf-a3f4-4c24-9aa0-f436dff0f296-openshift-service-ca\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:32:50.156125 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.156088    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/659f03bf-a3f4-4c24-9aa0-f436dff0f296-kube-api-access-zjk8c" (OuterVolumeSpecName: "kube-api-access-zjk8c") pod "659f03bf-a3f4-4c24-9aa0-f436dff0f296" (UID: "659f03bf-a3f4-4c24-9aa0-f436dff0f296"). InnerVolumeSpecName "kube-api-access-zjk8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:32:50.156375 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.156351    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/659f03bf-a3f4-4c24-9aa0-f436dff0f296-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "659f03bf-a3f4-4c24-9aa0-f436dff0f296" (UID: "659f03bf-a3f4-4c24-9aa0-f436dff0f296"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 18:32:50.255497 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.255463    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7d84819-6c7e-4b3b-8a16-97b9d99c6609-plugins-config-volume\") pod \"f7d84819-6c7e-4b3b-8a16-97b9d99c6609\" (UID: \"f7d84819-6c7e-4b3b-8a16-97b9d99c6609\") "
Jul 30 18:32:50.255669 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.255541    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6f64\" (UniqueName: \"kubernetes.io/projected/53c3ea19-33b3-4644-8b15-0daafaedad18-kube-api-access-x6f64\") pod \"53c3ea19-33b3-4644-8b15-0daafaedad18\" (UID: \"53c3ea19-33b3-4644-8b15-0daafaedad18\") "
Jul 30 18:32:50.255669 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.255584    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjtxn\" (UniqueName: \"kubernetes.io/projected/f7d84819-6c7e-4b3b-8a16-97b9d99c6609-kube-api-access-hjtxn\") pod \"f7d84819-6c7e-4b3b-8a16-97b9d99c6609\" (UID: \"f7d84819-6c7e-4b3b-8a16-97b9d99c6609\") "
Jul 30 18:32:50.255794 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.255668    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53c3ea19-33b3-4644-8b15-0daafaedad18-plugins-config-volume\") pod \"53c3ea19-33b3-4644-8b15-0daafaedad18\" (UID: \"53c3ea19-33b3-4644-8b15-0daafaedad18\") "
Jul 30 18:32:50.255901 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.255875    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7d84819-6c7e-4b3b-8a16-97b9d99c6609-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f7d84819-6c7e-4b3b-8a16-97b9d99c6609" (UID: "f7d84819-6c7e-4b3b-8a16-97b9d99c6609"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:32:50.256099 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.256077    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/659f03bf-a3f4-4c24-9aa0-f436dff0f296-maas-api-tls\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:32:50.256189 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.256102    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjk8c\" (UniqueName: \"kubernetes.io/projected/659f03bf-a3f4-4c24-9aa0-f436dff0f296-kube-api-access-zjk8c\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:32:50.256658 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.256631    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53c3ea19-33b3-4644-8b15-0daafaedad18-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "53c3ea19-33b3-4644-8b15-0daafaedad18" (UID: "53c3ea19-33b3-4644-8b15-0daafaedad18"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:32:50.257988 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.257952    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7d84819-6c7e-4b3b-8a16-97b9d99c6609-kube-api-access-hjtxn" (OuterVolumeSpecName: "kube-api-access-hjtxn") pod "f7d84819-6c7e-4b3b-8a16-97b9d99c6609" (UID: "f7d84819-6c7e-4b3b-8a16-97b9d99c6609"). InnerVolumeSpecName "kube-api-access-hjtxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:32:50.258370 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.258341    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53c3ea19-33b3-4644-8b15-0daafaedad18-kube-api-access-x6f64" (OuterVolumeSpecName: "kube-api-access-x6f64") pod "53c3ea19-33b3-4644-8b15-0daafaedad18" (UID: "53c3ea19-33b3-4644-8b15-0daafaedad18"). InnerVolumeSpecName "kube-api-access-x6f64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:32:50.264730 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.264661    2577 generic.go:356] "Generic (PLEG): container finished" podID="f7d84819-6c7e-4b3b-8a16-97b9d99c6609" containerID="6391fd6df9c8ed0e7565a0b5917320dd7d43aec858a29398f1525538a4a58830" exitCode=0
Jul 30 18:32:50.264730 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.264718    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk"
Jul 30 18:32:50.264939 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.264739    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk" event={"ID":"f7d84819-6c7e-4b3b-8a16-97b9d99c6609","Type":"ContainerDied","Data":"6391fd6df9c8ed0e7565a0b5917320dd7d43aec858a29398f1525538a4a58830"}
Jul 30 18:32:50.264939 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.264793    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk" event={"ID":"f7d84819-6c7e-4b3b-8a16-97b9d99c6609","Type":"ContainerDied","Data":"56e40b84931298c8a3040da9468d1f42f7f59a2a3745eb51d7a6979390668df4"}
Jul 30 18:32:50.264939 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.264853    2577 scope.go:117] "RemoveContainer" containerID="6391fd6df9c8ed0e7565a0b5917320dd7d43aec858a29398f1525538a4a58830"
Jul 30 18:32:50.266746 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.266713    2577 generic.go:356] "Generic (PLEG): container finished" podID="53c3ea19-33b3-4644-8b15-0daafaedad18" containerID="a2ac89b5c9045c3a1502a656b84219eafdac5d7331e7302d6576798c7f9f21c1" exitCode=0
Jul 30 18:32:50.266885 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.266767    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4"
Jul 30 18:32:50.266885 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.266787    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4" event={"ID":"53c3ea19-33b3-4644-8b15-0daafaedad18","Type":"ContainerDied","Data":"a2ac89b5c9045c3a1502a656b84219eafdac5d7331e7302d6576798c7f9f21c1"}
Jul 30 18:32:50.266885 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.266876    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4" event={"ID":"53c3ea19-33b3-4644-8b15-0daafaedad18","Type":"ContainerDied","Data":"8521b7b8aeec8c2ab5cc500f8d3c181fe9e896224595efe772daacb696b81a12"}
Jul 30 18:32:50.268344 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.268321    2577 generic.go:356] "Generic (PLEG): container finished" podID="659f03bf-a3f4-4c24-9aa0-f436dff0f296" containerID="301bde8adf98be1f26947c67c9ffc39b304b24e8ae73a65c912c7d4e188413d4" exitCode=0
Jul 30 18:32:50.268508 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.268355    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s" event={"ID":"659f03bf-a3f4-4c24-9aa0-f436dff0f296","Type":"ContainerDied","Data":"301bde8adf98be1f26947c67c9ffc39b304b24e8ae73a65c912c7d4e188413d4"}
Jul 30 18:32:50.268508 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.268391    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s" event={"ID":"659f03bf-a3f4-4c24-9aa0-f436dff0f296","Type":"ContainerDied","Data":"4cac60764941db6693f9b59d605302693d80363e06ef4a2250a454a6b3717abb"}
Jul 30 18:32:50.268508 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.268457    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s"
Jul 30 18:32:50.276529 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.276518    2577 scope.go:117] "RemoveContainer" containerID="6391fd6df9c8ed0e7565a0b5917320dd7d43aec858a29398f1525538a4a58830"
Jul 30 18:32:50.276864 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:32:50.276839    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6391fd6df9c8ed0e7565a0b5917320dd7d43aec858a29398f1525538a4a58830\": container with ID starting with 6391fd6df9c8ed0e7565a0b5917320dd7d43aec858a29398f1525538a4a58830 not found: ID does not exist" containerID="6391fd6df9c8ed0e7565a0b5917320dd7d43aec858a29398f1525538a4a58830"
Jul 30 18:32:50.276955 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.276874    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6391fd6df9c8ed0e7565a0b5917320dd7d43aec858a29398f1525538a4a58830"} err="failed to get container status \"6391fd6df9c8ed0e7565a0b5917320dd7d43aec858a29398f1525538a4a58830\": rpc error: code = NotFound desc = could not find container \"6391fd6df9c8ed0e7565a0b5917320dd7d43aec858a29398f1525538a4a58830\": container with ID starting with 6391fd6df9c8ed0e7565a0b5917320dd7d43aec858a29398f1525538a4a58830 not found: ID does not exist"
Jul 30 18:32:50.276955 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.276894    2577 scope.go:117] "RemoveContainer" containerID="a2ac89b5c9045c3a1502a656b84219eafdac5d7331e7302d6576798c7f9f21c1"
Jul 30 18:32:50.289607 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.289580    2577 scope.go:117] "RemoveContainer" containerID="a2ac89b5c9045c3a1502a656b84219eafdac5d7331e7302d6576798c7f9f21c1"
Jul 30 18:32:50.290133 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:32:50.290100    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2ac89b5c9045c3a1502a656b84219eafdac5d7331e7302d6576798c7f9f21c1\": container with ID starting with a2ac89b5c9045c3a1502a656b84219eafdac5d7331e7302d6576798c7f9f21c1 not found: ID does not exist" containerID="a2ac89b5c9045c3a1502a656b84219eafdac5d7331e7302d6576798c7f9f21c1"
Jul 30 18:32:50.290226 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.290146    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2ac89b5c9045c3a1502a656b84219eafdac5d7331e7302d6576798c7f9f21c1"} err="failed to get container status \"a2ac89b5c9045c3a1502a656b84219eafdac5d7331e7302d6576798c7f9f21c1\": rpc error: code = NotFound desc = could not find container \"a2ac89b5c9045c3a1502a656b84219eafdac5d7331e7302d6576798c7f9f21c1\": container with ID starting with a2ac89b5c9045c3a1502a656b84219eafdac5d7331e7302d6576798c7f9f21c1 not found: ID does not exist"
Jul 30 18:32:50.290226 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.290170    2577 scope.go:117] "RemoveContainer" containerID="301bde8adf98be1f26947c67c9ffc39b304b24e8ae73a65c912c7d4e188413d4"
Jul 30 18:32:50.291083 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.290551    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk"]
Jul 30 18:32:50.293877 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.293854    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fadb12-845c956c5d-qgrqk"]
Jul 30 18:32:50.299263 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.299243    2577 scope.go:117] "RemoveContainer" containerID="301bde8adf98be1f26947c67c9ffc39b304b24e8ae73a65c912c7d4e188413d4"
Jul 30 18:32:50.299530 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:32:50.299511    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"301bde8adf98be1f26947c67c9ffc39b304b24e8ae73a65c912c7d4e188413d4\": container with ID starting with 301bde8adf98be1f26947c67c9ffc39b304b24e8ae73a65c912c7d4e188413d4 not found: ID does not exist" containerID="301bde8adf98be1f26947c67c9ffc39b304b24e8ae73a65c912c7d4e188413d4"
Jul 30 18:32:50.299580 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.299537    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"301bde8adf98be1f26947c67c9ffc39b304b24e8ae73a65c912c7d4e188413d4"} err="failed to get container status \"301bde8adf98be1f26947c67c9ffc39b304b24e8ae73a65c912c7d4e188413d4\": rpc error: code = NotFound desc = could not find container \"301bde8adf98be1f26947c67c9ffc39b304b24e8ae73a65c912c7d4e188413d4\": container with ID starting with 301bde8adf98be1f26947c67c9ffc39b304b24e8ae73a65c912c7d4e188413d4 not found: ID does not exist"
Jul 30 18:32:50.304527 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.304502    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4"]
Jul 30 18:32:50.308506 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.308486    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fadb12-6b9f68955b-766b4"]
Jul 30 18:32:50.320010 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.319987    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s"]
Jul 30 18:32:50.325550 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.325528    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fadb12-566dc4455-8sq6s"]
Jul 30 18:32:50.357677 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.357636    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7d84819-6c7e-4b3b-8a16-97b9d99c6609-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:32:50.357677 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.357678    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6f64\" (UniqueName: \"kubernetes.io/projected/53c3ea19-33b3-4644-8b15-0daafaedad18-kube-api-access-x6f64\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:32:50.357875 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.357694    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjtxn\" (UniqueName: \"kubernetes.io/projected/f7d84819-6c7e-4b3b-8a16-97b9d99c6609-kube-api-access-hjtxn\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:32:50.357875 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.357707    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53c3ea19-33b3-4644-8b15-0daafaedad18-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:32:50.368735 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:50.368696    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:32:52.111677 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:52.111643    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53c3ea19-33b3-4644-8b15-0daafaedad18" path="/var/lib/kubelet/pods/53c3ea19-33b3-4644-8b15-0daafaedad18/volumes"
Jul 30 18:32:52.112096 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:52.112082    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="659f03bf-a3f4-4c24-9aa0-f436dff0f296" path="/var/lib/kubelet/pods/659f03bf-a3f4-4c24-9aa0-f436dff0f296/volumes"
Jul 30 18:32:52.112395 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:52.112383    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7d84819-6c7e-4b3b-8a16-97b9d99c6609" path="/var/lib/kubelet/pods/f7d84819-6c7e-4b3b-8a16-97b9d99c6609/volumes"
Jul 30 18:32:55.446694 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:55.446664    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:32:57.685326 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:32:57.685243    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:33:31.586018 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:31.585985    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:33:32.910293 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:32.910257    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:33:38.531451 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:38.531414    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:33:38.631961 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:38.631927    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:33:38.837113 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:38.837024    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:33:39.082877 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:39.082836    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:33:49.628908 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:49.628389    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8"]
Jul 30 18:33:49.628908 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:49.628664    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8" podUID="fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856" containerName="maas-api" containerID="cri-o://084e57df0e215aca592ed7dfaffa90510fb3a7dcbe1494e5ea586fc49fdc5bd6" gracePeriod=30
Jul 30 18:33:49.681471 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:49.681435    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc"]
Jul 30 18:33:49.681834 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:49.681692    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc" podUID="07ca1695-a48d-4d84-84aa-6839a06d3066" containerName="payload-processing" containerID="cri-o://d1cca186a602179ba880a91b585e1800ac6046057756689b0277b9da0e642e23" gracePeriod=30
Jul 30 18:33:49.789178 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:49.789143    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:33:49.844420 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:49.844380    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:33:49.935623 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:49.935594    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8"
Jul 30 18:33:49.977829 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:49.977788    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc"
Jul 30 18:33:50.049562 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.049526    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4qdr\" (UniqueName: \"kubernetes.io/projected/fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856-kube-api-access-f4qdr\") pod \"fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856\" (UID: \"fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856\") "
Jul 30 18:33:50.049749 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.049569    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856-openshift-service-ca\") pod \"fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856\" (UID: \"fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856\") "
Jul 30 18:33:50.049749 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.049626    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45qvz\" (UniqueName: \"kubernetes.io/projected/07ca1695-a48d-4d84-84aa-6839a06d3066-kube-api-access-45qvz\") pod \"07ca1695-a48d-4d84-84aa-6839a06d3066\" (UID: \"07ca1695-a48d-4d84-84aa-6839a06d3066\") "
Jul 30 18:33:50.049749 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.049658    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07ca1695-a48d-4d84-84aa-6839a06d3066-plugins-config-volume\") pod \"07ca1695-a48d-4d84-84aa-6839a06d3066\" (UID: \"07ca1695-a48d-4d84-84aa-6839a06d3066\") "
Jul 30 18:33:50.049749 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.049723    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856-maas-api-tls\") pod \"fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856\" (UID: \"fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856\") "
Jul 30 18:33:50.050052 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.050024    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856" (UID: "fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:33:50.050162 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.050129    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07ca1695-a48d-4d84-84aa-6839a06d3066-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "07ca1695-a48d-4d84-84aa-6839a06d3066" (UID: "07ca1695-a48d-4d84-84aa-6839a06d3066"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:33:50.051946 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.051925    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07ca1695-a48d-4d84-84aa-6839a06d3066-kube-api-access-45qvz" (OuterVolumeSpecName: "kube-api-access-45qvz") pod "07ca1695-a48d-4d84-84aa-6839a06d3066" (UID: "07ca1695-a48d-4d84-84aa-6839a06d3066"). InnerVolumeSpecName "kube-api-access-45qvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:33:50.052046 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.051943    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856-kube-api-access-f4qdr" (OuterVolumeSpecName: "kube-api-access-f4qdr") pod "fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856" (UID: "fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856"). InnerVolumeSpecName "kube-api-access-f4qdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:33:50.052046 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.052020    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856" (UID: "fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 18:33:50.150798 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.150712    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4qdr\" (UniqueName: \"kubernetes.io/projected/fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856-kube-api-access-f4qdr\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:33:50.150798 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.150748    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856-openshift-service-ca\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:33:50.150798 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.150762    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45qvz\" (UniqueName: \"kubernetes.io/projected/07ca1695-a48d-4d84-84aa-6839a06d3066-kube-api-access-45qvz\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:33:50.150798 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.150776    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07ca1695-a48d-4d84-84aa-6839a06d3066-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:33:50.150798 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.150789    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856-maas-api-tls\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:33:50.232075 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.232041    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:33:50.610855 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.610791    2577 generic.go:356] "Generic (PLEG): container finished" podID="07ca1695-a48d-4d84-84aa-6839a06d3066" containerID="d1cca186a602179ba880a91b585e1800ac6046057756689b0277b9da0e642e23" exitCode=0
Jul 30 18:33:50.610855 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.610860    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc"
Jul 30 18:33:50.611128 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.610880    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc" event={"ID":"07ca1695-a48d-4d84-84aa-6839a06d3066","Type":"ContainerDied","Data":"d1cca186a602179ba880a91b585e1800ac6046057756689b0277b9da0e642e23"}
Jul 30 18:33:50.611128 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.610944    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc" event={"ID":"07ca1695-a48d-4d84-84aa-6839a06d3066","Type":"ContainerDied","Data":"2a20ba68fa13f98059ce6dc0e6e3dec3671e33fc24e6976a86b260a753c080ce"}
Jul 30 18:33:50.611128 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.610974    2577 scope.go:117] "RemoveContainer" containerID="d1cca186a602179ba880a91b585e1800ac6046057756689b0277b9da0e642e23"
Jul 30 18:33:50.612350 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.612320    2577 generic.go:356] "Generic (PLEG): container finished" podID="fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856" containerID="084e57df0e215aca592ed7dfaffa90510fb3a7dcbe1494e5ea586fc49fdc5bd6" exitCode=0
Jul 30 18:33:50.612475 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.612361    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8" event={"ID":"fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856","Type":"ContainerDied","Data":"084e57df0e215aca592ed7dfaffa90510fb3a7dcbe1494e5ea586fc49fdc5bd6"}
Jul 30 18:33:50.612475 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.612377    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8"
Jul 30 18:33:50.612475 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.612402    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8" event={"ID":"fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856","Type":"ContainerDied","Data":"b15d20a3d8374d771db12238d6d1e5eb026731abda671c7381f2dbb108315221"}
Jul 30 18:33:50.622853 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.622827    2577 scope.go:117] "RemoveContainer" containerID="d1cca186a602179ba880a91b585e1800ac6046057756689b0277b9da0e642e23"
Jul 30 18:33:50.623188 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:33:50.623170    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1cca186a602179ba880a91b585e1800ac6046057756689b0277b9da0e642e23\": container with ID starting with d1cca186a602179ba880a91b585e1800ac6046057756689b0277b9da0e642e23 not found: ID does not exist" containerID="d1cca186a602179ba880a91b585e1800ac6046057756689b0277b9da0e642e23"
Jul 30 18:33:50.623251 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.623197    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1cca186a602179ba880a91b585e1800ac6046057756689b0277b9da0e642e23"} err="failed to get container status \"d1cca186a602179ba880a91b585e1800ac6046057756689b0277b9da0e642e23\": rpc error: code = NotFound desc = could not find container \"d1cca186a602179ba880a91b585e1800ac6046057756689b0277b9da0e642e23\": container with ID starting with d1cca186a602179ba880a91b585e1800ac6046057756689b0277b9da0e642e23 not found: ID does not exist"
Jul 30 18:33:50.623251 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.623214    2577 scope.go:117] "RemoveContainer" containerID="084e57df0e215aca592ed7dfaffa90510fb3a7dcbe1494e5ea586fc49fdc5bd6"
Jul 30 18:33:50.632005 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.631752    2577 scope.go:117] "RemoveContainer" containerID="084e57df0e215aca592ed7dfaffa90510fb3a7dcbe1494e5ea586fc49fdc5bd6"
Jul 30 18:33:50.632308 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:33:50.632032    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"084e57df0e215aca592ed7dfaffa90510fb3a7dcbe1494e5ea586fc49fdc5bd6\": container with ID starting with 084e57df0e215aca592ed7dfaffa90510fb3a7dcbe1494e5ea586fc49fdc5bd6 not found: ID does not exist" containerID="084e57df0e215aca592ed7dfaffa90510fb3a7dcbe1494e5ea586fc49fdc5bd6"
Jul 30 18:33:50.632308 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.632057    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"084e57df0e215aca592ed7dfaffa90510fb3a7dcbe1494e5ea586fc49fdc5bd6"} err="failed to get container status \"084e57df0e215aca592ed7dfaffa90510fb3a7dcbe1494e5ea586fc49fdc5bd6\": rpc error: code = NotFound desc = could not find container \"084e57df0e215aca592ed7dfaffa90510fb3a7dcbe1494e5ea586fc49fdc5bd6\": container with ID starting with 084e57df0e215aca592ed7dfaffa90510fb3a7dcbe1494e5ea586fc49fdc5bd6 not found: ID does not exist"
Jul 30 18:33:50.632516 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.632497    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8"]
Jul 30 18:33:50.636721 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.636697    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c8d7de-7694688549-9vgz8"]
Jul 30 18:33:50.647106 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.647076    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc"]
Jul 30 18:33:50.653568 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:50.653527    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c8d7de-66cb7bdd66-62mrc"]
Jul 30 18:33:52.111053 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:52.111016    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07ca1695-a48d-4d84-84aa-6839a06d3066" path="/var/lib/kubelet/pods/07ca1695-a48d-4d84-84aa-6839a06d3066/volumes"
Jul 30 18:33:52.111455 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:52.111441    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856" path="/var/lib/kubelet/pods/fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856/volumes"
Jul 30 18:33:56.062929 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:56.062875    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:33:56.121742 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:56.121706    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:33:56.364766 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:56.364675    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:33:56.609376 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:33:56.609343    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:34:07.153904 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.153870    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45"]
Jul 30 18:34:07.154417 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.154098    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45" podUID="15fc6fff-6986-4a20-96c8-ecdc35841c21" containerName="maas-api" containerID="cri-o://d3b64eb3e74ba23db4572fedd7a6614bc2c1c5f204458f692bd09a214ed74005" gracePeriod=30
Jul 30 18:34:07.186205 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.186172    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:34:07.214904 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.214861    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb"]
Jul 30 18:34:07.215149 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.215119    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb" podUID="9bf79219-bc3f-4e41-a044-f1f790051406" containerName="payload-processing" containerID="cri-o://54e858b6ca216bea68ead3230fc5cc312d1c97cefbde2e24e351cf5a7129bdbe" gracePeriod=30
Jul 30 18:34:07.223758 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.223666    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4"]
Jul 30 18:34:07.223940 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.223911    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4" podUID="925d2bea-64eb-4227-9b5a-8467d7507d92" containerName="payload-pre-processing" containerID="cri-o://aad44f32d189a0d90952d9eba5b4818c65c13e85d4cdacc0d63492d3779d80a1" gracePeriod=30
Jul 30 18:34:07.259191 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.259016    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:34:07.323975 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.323943    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:34:07.382036 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.382002    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:34:07.493869 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.493807    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45"
Jul 30 18:34:07.524997 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.524965    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb"
Jul 30 18:34:07.534552 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.534519    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4"
Jul 30 18:34:07.629887 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.629847    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9b6l\" (UniqueName: \"kubernetes.io/projected/15fc6fff-6986-4a20-96c8-ecdc35841c21-kube-api-access-j9b6l\") pod \"15fc6fff-6986-4a20-96c8-ecdc35841c21\" (UID: \"15fc6fff-6986-4a20-96c8-ecdc35841c21\") "
Jul 30 18:34:07.630095 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.629931    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15fc6fff-6986-4a20-96c8-ecdc35841c21-maas-api-tls\") pod \"15fc6fff-6986-4a20-96c8-ecdc35841c21\" (UID: \"15fc6fff-6986-4a20-96c8-ecdc35841c21\") "
Jul 30 18:34:07.630095 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.630026    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15fc6fff-6986-4a20-96c8-ecdc35841c21-openshift-service-ca\") pod \"15fc6fff-6986-4a20-96c8-ecdc35841c21\" (UID: \"15fc6fff-6986-4a20-96c8-ecdc35841c21\") "
Jul 30 18:34:07.630095 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.630065    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhzxq\" (UniqueName: \"kubernetes.io/projected/925d2bea-64eb-4227-9b5a-8467d7507d92-kube-api-access-hhzxq\") pod \"925d2bea-64eb-4227-9b5a-8467d7507d92\" (UID: \"925d2bea-64eb-4227-9b5a-8467d7507d92\") "
Jul 30 18:34:07.630095 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.630084    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7whf\" (UniqueName: \"kubernetes.io/projected/9bf79219-bc3f-4e41-a044-f1f790051406-kube-api-access-p7whf\") pod \"9bf79219-bc3f-4e41-a044-f1f790051406\" (UID: \"9bf79219-bc3f-4e41-a044-f1f790051406\") "
Jul 30 18:34:07.630327 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.630109    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/925d2bea-64eb-4227-9b5a-8467d7507d92-plugins-config-volume\") pod \"925d2bea-64eb-4227-9b5a-8467d7507d92\" (UID: \"925d2bea-64eb-4227-9b5a-8467d7507d92\") "
Jul 30 18:34:07.630327 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.630245    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bf79219-bc3f-4e41-a044-f1f790051406-plugins-config-volume\") pod \"9bf79219-bc3f-4e41-a044-f1f790051406\" (UID: \"9bf79219-bc3f-4e41-a044-f1f790051406\") "
Jul 30 18:34:07.630476 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.630450    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15fc6fff-6986-4a20-96c8-ecdc35841c21-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "15fc6fff-6986-4a20-96c8-ecdc35841c21" (UID: "15fc6fff-6986-4a20-96c8-ecdc35841c21"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:34:07.630538 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.630522    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/925d2bea-64eb-4227-9b5a-8467d7507d92-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "925d2bea-64eb-4227-9b5a-8467d7507d92" (UID: "925d2bea-64eb-4227-9b5a-8467d7507d92"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:34:07.630699 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.630679    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15fc6fff-6986-4a20-96c8-ecdc35841c21-openshift-service-ca\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:34:07.630889 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.630705    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/925d2bea-64eb-4227-9b5a-8467d7507d92-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:34:07.631005 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.630978    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bf79219-bc3f-4e41-a044-f1f790051406-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9bf79219-bc3f-4e41-a044-f1f790051406" (UID: "9bf79219-bc3f-4e41-a044-f1f790051406"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:34:07.632367 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.632347    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bf79219-bc3f-4e41-a044-f1f790051406-kube-api-access-p7whf" (OuterVolumeSpecName: "kube-api-access-p7whf") pod "9bf79219-bc3f-4e41-a044-f1f790051406" (UID: "9bf79219-bc3f-4e41-a044-f1f790051406"). InnerVolumeSpecName "kube-api-access-p7whf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:34:07.632476 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.632374    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15fc6fff-6986-4a20-96c8-ecdc35841c21-kube-api-access-j9b6l" (OuterVolumeSpecName: "kube-api-access-j9b6l") pod "15fc6fff-6986-4a20-96c8-ecdc35841c21" (UID: "15fc6fff-6986-4a20-96c8-ecdc35841c21"). InnerVolumeSpecName "kube-api-access-j9b6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:34:07.632553 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.632540    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/925d2bea-64eb-4227-9b5a-8467d7507d92-kube-api-access-hhzxq" (OuterVolumeSpecName: "kube-api-access-hhzxq") pod "925d2bea-64eb-4227-9b5a-8467d7507d92" (UID: "925d2bea-64eb-4227-9b5a-8467d7507d92"). InnerVolumeSpecName "kube-api-access-hhzxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:34:07.632604 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.632586    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15fc6fff-6986-4a20-96c8-ecdc35841c21-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "15fc6fff-6986-4a20-96c8-ecdc35841c21" (UID: "15fc6fff-6986-4a20-96c8-ecdc35841c21"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 18:34:07.706320 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.706231    2577 generic.go:356] "Generic (PLEG): container finished" podID="925d2bea-64eb-4227-9b5a-8467d7507d92" containerID="aad44f32d189a0d90952d9eba5b4818c65c13e85d4cdacc0d63492d3779d80a1" exitCode=0
Jul 30 18:34:07.706320 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.706285    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4"
Jul 30 18:34:07.706541 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.706315    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4" event={"ID":"925d2bea-64eb-4227-9b5a-8467d7507d92","Type":"ContainerDied","Data":"aad44f32d189a0d90952d9eba5b4818c65c13e85d4cdacc0d63492d3779d80a1"}
Jul 30 18:34:07.706541 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.706387    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4" event={"ID":"925d2bea-64eb-4227-9b5a-8467d7507d92","Type":"ContainerDied","Data":"b7b36acd3981f1bfceed90eafec9d0e70788ac74c97ad58f9a9e3cc91ae074ad"}
Jul 30 18:34:07.706541 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.706421    2577 scope.go:117] "RemoveContainer" containerID="aad44f32d189a0d90952d9eba5b4818c65c13e85d4cdacc0d63492d3779d80a1"
Jul 30 18:34:07.707777 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.707662    2577 generic.go:356] "Generic (PLEG): container finished" podID="15fc6fff-6986-4a20-96c8-ecdc35841c21" containerID="d3b64eb3e74ba23db4572fedd7a6614bc2c1c5f204458f692bd09a214ed74005" exitCode=0
Jul 30 18:34:07.707777 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.707698    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45" event={"ID":"15fc6fff-6986-4a20-96c8-ecdc35841c21","Type":"ContainerDied","Data":"d3b64eb3e74ba23db4572fedd7a6614bc2c1c5f204458f692bd09a214ed74005"}
Jul 30 18:34:07.707777 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.707721    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45"
Jul 30 18:34:07.707777 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.707751    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45" event={"ID":"15fc6fff-6986-4a20-96c8-ecdc35841c21","Type":"ContainerDied","Data":"1bad3ad529fe91d7fa4cd0b01e9a80eecb0fdc58e9c2383a8cca8a415548e03d"}
Jul 30 18:34:07.709211 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.709187    2577 generic.go:356] "Generic (PLEG): container finished" podID="9bf79219-bc3f-4e41-a044-f1f790051406" containerID="54e858b6ca216bea68ead3230fc5cc312d1c97cefbde2e24e351cf5a7129bdbe" exitCode=0
Jul 30 18:34:07.709354 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.709218    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb"
Jul 30 18:34:07.709354 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.709253    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb" event={"ID":"9bf79219-bc3f-4e41-a044-f1f790051406","Type":"ContainerDied","Data":"54e858b6ca216bea68ead3230fc5cc312d1c97cefbde2e24e351cf5a7129bdbe"}
Jul 30 18:34:07.709354 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.709296    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb" event={"ID":"9bf79219-bc3f-4e41-a044-f1f790051406","Type":"ContainerDied","Data":"1fdcf2e49f5c2261f1e96a2a27ebedb74ba33e788dae6ff2d5d4194a103e34b6"}
Jul 30 18:34:07.718606 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.718585    2577 scope.go:117] "RemoveContainer" containerID="aad44f32d189a0d90952d9eba5b4818c65c13e85d4cdacc0d63492d3779d80a1"
Jul 30 18:34:07.718946 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:34:07.718920    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aad44f32d189a0d90952d9eba5b4818c65c13e85d4cdacc0d63492d3779d80a1\": container with ID starting with aad44f32d189a0d90952d9eba5b4818c65c13e85d4cdacc0d63492d3779d80a1 not found: ID does not exist" containerID="aad44f32d189a0d90952d9eba5b4818c65c13e85d4cdacc0d63492d3779d80a1"
Jul 30 18:34:07.719099 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.718959    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aad44f32d189a0d90952d9eba5b4818c65c13e85d4cdacc0d63492d3779d80a1"} err="failed to get container status \"aad44f32d189a0d90952d9eba5b4818c65c13e85d4cdacc0d63492d3779d80a1\": rpc error: code = NotFound desc = could not find container \"aad44f32d189a0d90952d9eba5b4818c65c13e85d4cdacc0d63492d3779d80a1\": container with ID starting with aad44f32d189a0d90952d9eba5b4818c65c13e85d4cdacc0d63492d3779d80a1 not found: ID does not exist"
Jul 30 18:34:07.719099 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.719029    2577 scope.go:117] "RemoveContainer" containerID="d3b64eb3e74ba23db4572fedd7a6614bc2c1c5f204458f692bd09a214ed74005"
Jul 30 18:34:07.727798 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.727780    2577 scope.go:117] "RemoveContainer" containerID="d3b64eb3e74ba23db4572fedd7a6614bc2c1c5f204458f692bd09a214ed74005"
Jul 30 18:34:07.728108 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:34:07.728092    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3b64eb3e74ba23db4572fedd7a6614bc2c1c5f204458f692bd09a214ed74005\": container with ID starting with d3b64eb3e74ba23db4572fedd7a6614bc2c1c5f204458f692bd09a214ed74005 not found: ID does not exist" containerID="d3b64eb3e74ba23db4572fedd7a6614bc2c1c5f204458f692bd09a214ed74005"
Jul 30 18:34:07.728153 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.728115    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3b64eb3e74ba23db4572fedd7a6614bc2c1c5f204458f692bd09a214ed74005"} err="failed to get container status \"d3b64eb3e74ba23db4572fedd7a6614bc2c1c5f204458f692bd09a214ed74005\": rpc error: code = NotFound desc = could not find container \"d3b64eb3e74ba23db4572fedd7a6614bc2c1c5f204458f692bd09a214ed74005\": container with ID starting with d3b64eb3e74ba23db4572fedd7a6614bc2c1c5f204458f692bd09a214ed74005 not found: ID does not exist"
Jul 30 18:34:07.728153 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.728130    2577 scope.go:117] "RemoveContainer" containerID="54e858b6ca216bea68ead3230fc5cc312d1c97cefbde2e24e351cf5a7129bdbe"
Jul 30 18:34:07.732115 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.732093    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4"]
Jul 30 18:34:07.732224 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.732205    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bf79219-bc3f-4e41-a044-f1f790051406-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:34:07.732224 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.732222    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9b6l\" (UniqueName: \"kubernetes.io/projected/15fc6fff-6986-4a20-96c8-ecdc35841c21-kube-api-access-j9b6l\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:34:07.732322 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.732232    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15fc6fff-6986-4a20-96c8-ecdc35841c21-maas-api-tls\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:34:07.732322 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.732241    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhzxq\" (UniqueName: \"kubernetes.io/projected/925d2bea-64eb-4227-9b5a-8467d7507d92-kube-api-access-hhzxq\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:34:07.732322 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.732249    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7whf\" (UniqueName: \"kubernetes.io/projected/9bf79219-bc3f-4e41-a044-f1f790051406-kube-api-access-p7whf\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:34:07.736106 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.736085    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-8f586d-868fc85f76-zqbm4"]
Jul 30 18:34:07.736798 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.736782    2577 scope.go:117] "RemoveContainer" containerID="54e858b6ca216bea68ead3230fc5cc312d1c97cefbde2e24e351cf5a7129bdbe"
Jul 30 18:34:07.737101 ip-10-0-135-203 kubenswrapper[2577]: E0730 18:34:07.737084    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54e858b6ca216bea68ead3230fc5cc312d1c97cefbde2e24e351cf5a7129bdbe\": container with ID starting with 54e858b6ca216bea68ead3230fc5cc312d1c97cefbde2e24e351cf5a7129bdbe not found: ID does not exist" containerID="54e858b6ca216bea68ead3230fc5cc312d1c97cefbde2e24e351cf5a7129bdbe"
Jul 30 18:34:07.737184 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.737107    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54e858b6ca216bea68ead3230fc5cc312d1c97cefbde2e24e351cf5a7129bdbe"} err="failed to get container status \"54e858b6ca216bea68ead3230fc5cc312d1c97cefbde2e24e351cf5a7129bdbe\": rpc error: code = NotFound desc = could not find container \"54e858b6ca216bea68ead3230fc5cc312d1c97cefbde2e24e351cf5a7129bdbe\": container with ID starting with 54e858b6ca216bea68ead3230fc5cc312d1c97cefbde2e24e351cf5a7129bdbe not found: ID does not exist"
Jul 30 18:34:07.745284 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.745264    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb"]
Jul 30 18:34:07.749449 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.749425    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-8f586d-6f6c9f896b-gh9zb"]
Jul 30 18:34:07.760648 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.760620    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45"]
Jul 30 18:34:07.765245 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.765223    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:34:07.765537 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:07.765518    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8f586d-5455dd9cc4-7xl45"]
Jul 30 18:34:08.111486 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:08.111449    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15fc6fff-6986-4a20-96c8-ecdc35841c21" path="/var/lib/kubelet/pods/15fc6fff-6986-4a20-96c8-ecdc35841c21/volumes"
Jul 30 18:34:08.111863 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:08.111849    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="925d2bea-64eb-4227-9b5a-8467d7507d92" path="/var/lib/kubelet/pods/925d2bea-64eb-4227-9b5a-8467d7507d92/volumes"
Jul 30 18:34:08.112169 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:08.112157    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bf79219-bc3f-4e41-a044-f1f790051406" path="/var/lib/kubelet/pods/9bf79219-bc3f-4e41-a044-f1f790051406/volumes"
Jul 30 18:34:13.850613 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:13.850579    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:34:13.951288 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:13.951251    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:34:14.153145 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:14.153060    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:34:14.385207 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:14.385172    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:34:19.563056 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.563014    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg"]
Jul 30 18:34:19.563935 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.563881    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg" podUID="b9518ffe-8b29-4784-854e-3a31b71e8624" containerName="maas-api" containerID="cri-o://fa047225b144e5115ae1923ca4360bb960263ddacd2078728376f4b8bc22782d" gracePeriod=30
Jul 30 18:34:19.583804 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.583778    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:34:19.622135 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.622021    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp"]
Jul 30 18:34:19.622341 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.622269    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp" podUID="5dc55636-aca9-48d8-8ba1-8c6110c262f7" containerName="payload-pre-processing" containerID="cri-o://6b3bdd2aa6a6227e3a1cd700b2a48403253ded894f4ada49c697903752f555b2" gracePeriod=30
Jul 30 18:34:19.681597 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.681565    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:34:19.764154 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.764077    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:34:19.783156 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.783090    2577 generic.go:356] "Generic (PLEG): container finished" podID="5dc55636-aca9-48d8-8ba1-8c6110c262f7" containerID="6b3bdd2aa6a6227e3a1cd700b2a48403253ded894f4ada49c697903752f555b2" exitCode=0
Jul 30 18:34:19.783353 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.783163    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp" event={"ID":"5dc55636-aca9-48d8-8ba1-8c6110c262f7","Type":"ContainerDied","Data":"6b3bdd2aa6a6227e3a1cd700b2a48403253ded894f4ada49c697903752f555b2"}
Jul 30 18:34:19.784981 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.784960    2577 generic.go:356] "Generic (PLEG): container finished" podID="b9518ffe-8b29-4784-854e-3a31b71e8624" containerID="fa047225b144e5115ae1923ca4360bb960263ddacd2078728376f4b8bc22782d" exitCode=0
Jul 30 18:34:19.785115 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.785034    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg" event={"ID":"b9518ffe-8b29-4784-854e-3a31b71e8624","Type":"ContainerDied","Data":"fa047225b144e5115ae1923ca4360bb960263ddacd2078728376f4b8bc22782d"}
Jul 30 18:34:19.877660 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.877633    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg"
Jul 30 18:34:19.887285 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.887247    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:34:19.908724 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.908696    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp"
Jul 30 18:34:19.953300 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.953265    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9518ffe-8b29-4784-854e-3a31b71e8624-openshift-service-ca\") pod \"b9518ffe-8b29-4784-854e-3a31b71e8624\" (UID: \"b9518ffe-8b29-4784-854e-3a31b71e8624\") "
Jul 30 18:34:19.953482 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.953350    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwrjd\" (UniqueName: \"kubernetes.io/projected/5dc55636-aca9-48d8-8ba1-8c6110c262f7-kube-api-access-hwrjd\") pod \"5dc55636-aca9-48d8-8ba1-8c6110c262f7\" (UID: \"5dc55636-aca9-48d8-8ba1-8c6110c262f7\") "
Jul 30 18:34:19.953482 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.953377    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9518ffe-8b29-4784-854e-3a31b71e8624-maas-api-tls\") pod \"b9518ffe-8b29-4784-854e-3a31b71e8624\" (UID: \"b9518ffe-8b29-4784-854e-3a31b71e8624\") "
Jul 30 18:34:19.953482 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.953396    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96m75\" (UniqueName: \"kubernetes.io/projected/b9518ffe-8b29-4784-854e-3a31b71e8624-kube-api-access-96m75\") pod \"b9518ffe-8b29-4784-854e-3a31b71e8624\" (UID: \"b9518ffe-8b29-4784-854e-3a31b71e8624\") "
Jul 30 18:34:19.953634 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.953564    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5dc55636-aca9-48d8-8ba1-8c6110c262f7-plugins-config-volume\") pod \"5dc55636-aca9-48d8-8ba1-8c6110c262f7\" (UID: \"5dc55636-aca9-48d8-8ba1-8c6110c262f7\") "
Jul 30 18:34:19.953634 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.953598    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9518ffe-8b29-4784-854e-3a31b71e8624-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b9518ffe-8b29-4784-854e-3a31b71e8624" (UID: "b9518ffe-8b29-4784-854e-3a31b71e8624"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:34:19.953929 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.953901    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9518ffe-8b29-4784-854e-3a31b71e8624-openshift-service-ca\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:34:19.954001 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.953952    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dc55636-aca9-48d8-8ba1-8c6110c262f7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5dc55636-aca9-48d8-8ba1-8c6110c262f7" (UID: "5dc55636-aca9-48d8-8ba1-8c6110c262f7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 18:34:19.955633 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.955605    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9518ffe-8b29-4784-854e-3a31b71e8624-kube-api-access-96m75" (OuterVolumeSpecName: "kube-api-access-96m75") pod "b9518ffe-8b29-4784-854e-3a31b71e8624" (UID: "b9518ffe-8b29-4784-854e-3a31b71e8624"). InnerVolumeSpecName "kube-api-access-96m75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:34:19.955633 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.955610    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9518ffe-8b29-4784-854e-3a31b71e8624-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b9518ffe-8b29-4784-854e-3a31b71e8624" (UID: "b9518ffe-8b29-4784-854e-3a31b71e8624"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 18:34:19.955744 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:19.955666    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dc55636-aca9-48d8-8ba1-8c6110c262f7-kube-api-access-hwrjd" (OuterVolumeSpecName: "kube-api-access-hwrjd") pod "5dc55636-aca9-48d8-8ba1-8c6110c262f7" (UID: "5dc55636-aca9-48d8-8ba1-8c6110c262f7"). InnerVolumeSpecName "kube-api-access-hwrjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 18:34:20.054699 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:20.054666    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwrjd\" (UniqueName: \"kubernetes.io/projected/5dc55636-aca9-48d8-8ba1-8c6110c262f7-kube-api-access-hwrjd\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:34:20.054699 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:20.054696    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9518ffe-8b29-4784-854e-3a31b71e8624-maas-api-tls\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:34:20.054699 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:20.054707    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96m75\" (UniqueName: \"kubernetes.io/projected/b9518ffe-8b29-4784-854e-3a31b71e8624-kube-api-access-96m75\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:34:20.054990 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:20.054718    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5dc55636-aca9-48d8-8ba1-8c6110c262f7-plugins-config-volume\") on node \"ip-10-0-135-203.ec2.internal\" DevicePath \"\""
Jul 30 18:34:20.152578 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:20.152528    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:34:20.791566 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:20.791537    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp"
Jul 30 18:34:20.791566 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:20.791563    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp" event={"ID":"5dc55636-aca9-48d8-8ba1-8c6110c262f7","Type":"ContainerDied","Data":"36f0c107ca4775a6f031db5d799b2b0326549db7dc9e52da4ac975d54e99382d"}
Jul 30 18:34:20.792150 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:20.791631    2577 scope.go:117] "RemoveContainer" containerID="6b3bdd2aa6a6227e3a1cd700b2a48403253ded894f4ada49c697903752f555b2"
Jul 30 18:34:20.793177 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:20.793151    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg"
Jul 30 18:34:20.793177 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:20.793198    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg" event={"ID":"b9518ffe-8b29-4784-854e-3a31b71e8624","Type":"ContainerDied","Data":"3f31fba30bbabbc8fd9aa37fc464c95b51543992e411371a56e9dd16a128c625"}
Jul 30 18:34:20.801185 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:20.801161    2577 scope.go:117] "RemoveContainer" containerID="fa047225b144e5115ae1923ca4360bb960263ddacd2078728376f4b8bc22782d"
Jul 30 18:34:20.812023 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:20.812001    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg"]
Jul 30 18:34:20.815784 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:20.815763    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fbbaa9-6fc7986c7f-hbwpg"]
Jul 30 18:34:20.826460 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:20.826437    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp"]
Jul 30 18:34:20.829866 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:20.829847    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-fbbaa9-67bd78c7d8-5r5lp"]
Jul 30 18:34:22.111712 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:22.111683    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dc55636-aca9-48d8-8ba1-8c6110c262f7" path="/var/lib/kubelet/pods/5dc55636-aca9-48d8-8ba1-8c6110c262f7/volumes"
Jul 30 18:34:22.112165 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:22.112149    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9518ffe-8b29-4784-854e-3a31b71e8624" path="/var/lib/kubelet/pods/b9518ffe-8b29-4784-854e-3a31b71e8624/volumes"
Jul 30 18:34:23.281776 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:23.281736    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:34:26.211322 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:26.211244    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:34:26.265978 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:26.265948    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:34:26.491693 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:26.491561    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:34:26.734518 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:26.734477    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:34:33.014933 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:33.014899    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-mchct_b1596812-1f98-42ab-ace0-01fd85f11a4c/manager/0.log"
Jul 30 18:34:33.146033 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:33.145991    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-98m6c_421c5189-15a2-4509-9c61-2ffbfdd35204/manager/0.log"
Jul 30 18:34:33.274213 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:33.274138    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-59fc4686ff-gfw5x_170f7d7d-61e4-4269-8884-fae812916c3e/manager/0.log"
Jul 30 18:34:33.391093 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:33.391067    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-vxzgh_e86cc99c-4b17-4534-845a-73c8d130dba9/server/0.log"
Jul 30 18:34:33.512285 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:33.512251    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-jtrxh_29a8ca15-2df6-4ba4-adfc-a0024f4c3648/manager/0.log"
Jul 30 18:34:33.639006 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:33.638921    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-975dfc96f-6pz5c_9847f8a2-f792-433a-8e4e-25cad18e13d2/copy-manifests/0.log"
Jul 30 18:34:33.652377 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:33.652344    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-975dfc96f-6pz5c_9847f8a2-f792-433a-8e4e-25cad18e13d2/manager/0.log"
Jul 30 18:34:34.297929 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:34.297902    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-ff54b6d89-67f82_cdae6d27-dd7b-4f7c-b92c-50282580fb2d/maas-api/0.log"
Jul 30 18:34:34.421004 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:34.420979    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29757240-8fwlc_384eb3e0-9877-4a95-975a-49016d7a7fe0/cleanup/0.log"
Jul 30 18:34:34.772949 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:34.772924    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-75knz_f55376ff-9026-491a-9a91-563ec1ec77ea/postgres/0.log"
Jul 30 18:34:36.365405 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:36.365375    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-vgzwj_a5d7663e-d9ab-410e-9759-7477ba5c5f90/manager/0.log"
Jul 30 18:34:36.485395 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:36.485368    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-lblst_2c73c8dc-987b-4121-88f7-c7a90683936a/manager/0.log"
Jul 30 18:34:36.729780 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:36.729705    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-95lpf_5237a970-40b8-4b10-b4a1-237e27d0a2af/registry-server/0.log"
Jul 30 18:34:36.862525 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:36.862493    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-xmfjp_f07c466d-53a4-4416-94a3-4bef276fb42f/manager/0.log"
Jul 30 18:34:37.117971 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:37.117939    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-sjv69_73e3ca24-3939-4b09-89af-01b8100030cd/manager/0.log"
Jul 30 18:34:37.836662 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:37.836633    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6ff87f57bc-md8k2_19d2cd9d-54a4-44d9-b08c-45c3afa59146/kube-auth-proxy/0.log"
Jul 30 18:34:38.216991 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:38.216895    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-t25kk_335be1a6-121e-41a2-b390-180e8c87ccdf/payload-processing/0.log"
Jul 30 18:34:38.687396 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:38.687355    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-rddbj_17ac034f-cbc3-49de-98c1-0c0bcf06d098/main/0.log"
Jul 30 18:34:39.063727 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:39.063695    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4fx22_887b8ea5-c7a7-42c2-8d46-b4df91b4b477/main/0.log"
Jul 30 18:34:45.579377 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.579333    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vp2v8/must-gather-x5n7t"]
Jul 30 18:34:45.579983 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.579964    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bf79219-bc3f-4e41-a044-f1f790051406" containerName="payload-processing"
Jul 30 18:34:45.579983 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.579985    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bf79219-bc3f-4e41-a044-f1f790051406" containerName="payload-processing"
Jul 30 18:34:45.580141 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580000    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9518ffe-8b29-4784-854e-3a31b71e8624" containerName="maas-api"
Jul 30 18:34:45.580141 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580007    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9518ffe-8b29-4784-854e-3a31b71e8624" containerName="maas-api"
Jul 30 18:34:45.580141 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580024    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcdf4e0d-1915-44b2-ad00-374338dd1e90" containerName="maas-api"
Jul 30 18:34:45.580141 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580033    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcdf4e0d-1915-44b2-ad00-374338dd1e90" containerName="maas-api"
Jul 30 18:34:45.580141 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580061    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856" containerName="maas-api"
Jul 30 18:34:45.580141 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580068    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856" containerName="maas-api"
Jul 30 18:34:45.580141 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580082    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15fc6fff-6986-4a20-96c8-ecdc35841c21" containerName="maas-api"
Jul 30 18:34:45.580141 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580089    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="15fc6fff-6986-4a20-96c8-ecdc35841c21" containerName="maas-api"
Jul 30 18:34:45.580141 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580098    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="925d2bea-64eb-4227-9b5a-8467d7507d92" containerName="payload-pre-processing"
Jul 30 18:34:45.580141 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580106    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="925d2bea-64eb-4227-9b5a-8467d7507d92" containerName="payload-pre-processing"
Jul 30 18:34:45.580141 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580116    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dc55636-aca9-48d8-8ba1-8c6110c262f7" containerName="payload-pre-processing"
Jul 30 18:34:45.580141 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580124    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dc55636-aca9-48d8-8ba1-8c6110c262f7" containerName="payload-pre-processing"
Jul 30 18:34:45.580141 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580134    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53c3ea19-33b3-4644-8b15-0daafaedad18" containerName="payload-pre-processing"
Jul 30 18:34:45.580141 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580143    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="53c3ea19-33b3-4644-8b15-0daafaedad18" containerName="payload-pre-processing"
Jul 30 18:34:45.580849 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580152    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07ca1695-a48d-4d84-84aa-6839a06d3066" containerName="payload-processing"
Jul 30 18:34:45.580849 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580160    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="07ca1695-a48d-4d84-84aa-6839a06d3066" containerName="payload-processing"
Jul 30 18:34:45.580849 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580171    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="659f03bf-a3f4-4c24-9aa0-f436dff0f296" containerName="maas-api"
Jul 30 18:34:45.580849 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580179    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="659f03bf-a3f4-4c24-9aa0-f436dff0f296" containerName="maas-api"
Jul 30 18:34:45.580849 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580192    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7d84819-6c7e-4b3b-8a16-97b9d99c6609" containerName="payload-processing"
Jul 30 18:34:45.580849 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580200    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7d84819-6c7e-4b3b-8a16-97b9d99c6609" containerName="payload-processing"
Jul 30 18:34:45.580849 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580330    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="15fc6fff-6986-4a20-96c8-ecdc35841c21" containerName="maas-api"
Jul 30 18:34:45.580849 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580345    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bf79219-bc3f-4e41-a044-f1f790051406" containerName="payload-processing"
Jul 30 18:34:45.580849 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580355    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="53c3ea19-33b3-4644-8b15-0daafaedad18" containerName="payload-pre-processing"
Jul 30 18:34:45.580849 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580367    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="925d2bea-64eb-4227-9b5a-8467d7507d92" containerName="payload-pre-processing"
Jul 30 18:34:45.580849 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580376    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9518ffe-8b29-4784-854e-3a31b71e8624" containerName="maas-api"
Jul 30 18:34:45.580849 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580384    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7d84819-6c7e-4b3b-8a16-97b9d99c6609" containerName="payload-processing"
Jul 30 18:34:45.580849 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580393    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="659f03bf-a3f4-4c24-9aa0-f436dff0f296" containerName="maas-api"
Jul 30 18:34:45.580849 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580400    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcdf4e0d-1915-44b2-ad00-374338dd1e90" containerName="maas-api"
Jul 30 18:34:45.580849 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580410    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcc9bd89-5e68-4ed5-b6ad-ef7dde6b6856" containerName="maas-api"
Jul 30 18:34:45.580849 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580419    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dc55636-aca9-48d8-8ba1-8c6110c262f7" containerName="payload-pre-processing"
Jul 30 18:34:45.580849 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.580428    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="07ca1695-a48d-4d84-84aa-6839a06d3066" containerName="payload-processing"
Jul 30 18:34:45.585425 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.585404    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vp2v8/must-gather-x5n7t"
Jul 30 18:34:45.587640 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.587611    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vp2v8\"/\"kube-root-ca.crt\""
Jul 30 18:34:45.587766 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.587685    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vp2v8\"/\"default-dockercfg-pj59w\""
Jul 30 18:34:45.587766 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.587700    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vp2v8\"/\"openshift-service-ca.crt\""
Jul 30 18:34:45.591647 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.591622    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vp2v8/must-gather-x5n7t"]
Jul 30 18:34:45.598833 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.598783    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:34:45.684713 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.684673    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7bc6a3c4-ee58-4538-a429-6ed8e734e3c8-must-gather-output\") pod \"must-gather-x5n7t\" (UID: \"7bc6a3c4-ee58-4538-a429-6ed8e734e3c8\") " pod="openshift-must-gather-vp2v8/must-gather-x5n7t"
Jul 30 18:34:45.684713 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.684722    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp8sm\" (UniqueName: \"kubernetes.io/projected/7bc6a3c4-ee58-4538-a429-6ed8e734e3c8-kube-api-access-dp8sm\") pod \"must-gather-x5n7t\" (UID: \"7bc6a3c4-ee58-4538-a429-6ed8e734e3c8\") " pod="openshift-must-gather-vp2v8/must-gather-x5n7t"
Jul 30 18:34:45.786247 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.786210    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7bc6a3c4-ee58-4538-a429-6ed8e734e3c8-must-gather-output\") pod \"must-gather-x5n7t\" (UID: \"7bc6a3c4-ee58-4538-a429-6ed8e734e3c8\") " pod="openshift-must-gather-vp2v8/must-gather-x5n7t"
Jul 30 18:34:45.786247 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.786252    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp8sm\" (UniqueName: \"kubernetes.io/projected/7bc6a3c4-ee58-4538-a429-6ed8e734e3c8-kube-api-access-dp8sm\") pod \"must-gather-x5n7t\" (UID: \"7bc6a3c4-ee58-4538-a429-6ed8e734e3c8\") " pod="openshift-must-gather-vp2v8/must-gather-x5n7t"
Jul 30 18:34:45.786620 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.786596    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7bc6a3c4-ee58-4538-a429-6ed8e734e3c8-must-gather-output\") pod \"must-gather-x5n7t\" (UID: \"7bc6a3c4-ee58-4538-a429-6ed8e734e3c8\") " pod="openshift-must-gather-vp2v8/must-gather-x5n7t"
Jul 30 18:34:45.794201 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.794173    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp8sm\" (UniqueName: \"kubernetes.io/projected/7bc6a3c4-ee58-4538-a429-6ed8e734e3c8-kube-api-access-dp8sm\") pod \"must-gather-x5n7t\" (UID: \"7bc6a3c4-ee58-4538-a429-6ed8e734e3c8\") " pod="openshift-must-gather-vp2v8/must-gather-x5n7t"
Jul 30 18:34:45.895634 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:45.895552    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vp2v8/must-gather-x5n7t"
Jul 30 18:34:46.034425 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:46.034393    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vp2v8/must-gather-x5n7t"]
Jul 30 18:34:46.036459 ip-10-0-135-203 kubenswrapper[2577]: W0730 18:34:46.036430    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7bc6a3c4_ee58_4538_a429_6ed8e734e3c8.slice/crio-85d951766e297c4a8cb6c4aae3ff90c233d5bbf434ce10c5ff2d6b213efb9221 WatchSource:0}: Error finding container 85d951766e297c4a8cb6c4aae3ff90c233d5bbf434ce10c5ff2d6b213efb9221: Status 404 returned error can't find the container with id 85d951766e297c4a8cb6c4aae3ff90c233d5bbf434ce10c5ff2d6b213efb9221
Jul 30 18:34:46.038411 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:46.038392    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 18:34:46.940040 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:46.939974    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vp2v8/must-gather-x5n7t" event={"ID":"7bc6a3c4-ee58-4538-a429-6ed8e734e3c8","Type":"ContainerStarted","Data":"0621dd36817d9b69aae801233bb380e904583488707a218e6e5c5551e81f4e6f"}
Jul 30 18:34:46.940040 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:46.940015    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vp2v8/must-gather-x5n7t" event={"ID":"7bc6a3c4-ee58-4538-a429-6ed8e734e3c8","Type":"ContainerStarted","Data":"85d951766e297c4a8cb6c4aae3ff90c233d5bbf434ce10c5ff2d6b213efb9221"}
Jul 30 18:34:47.284277 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:47.284228    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log" limit=1024
Jul 30 18:34:47.951137 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:47.951099    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vp2v8/must-gather-x5n7t" event={"ID":"7bc6a3c4-ee58-4538-a429-6ed8e734e3c8","Type":"ContainerStarted","Data":"bca83dc71faac9a030c69d8388e23d7d36ff7aceff2c7b96414f151e3280d2c8"}
Jul 30 18:34:47.968768 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:47.968717    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vp2v8/must-gather-x5n7t" podStartSLOduration=2.187632847 podStartE2EDuration="2.968701795s" totalImagesPullingTime="781.068948ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 18:34:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:34:47.966438643 +0000 UTC m=+4062.420002635" watchObservedRunningTime="2026-07-30 18:34:47.968701795 +0000 UTC m=+4062.422265786"
Jul 30 18:34:48.512911 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:48.512876    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-f4vp7_a3251e0d-bc32-44a8-848d-2e64df31b7cc/global-pull-secret-syncer/0.log"
Jul 30 18:34:48.568413 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:48.568380    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6wttc_8cdacb83-3cbb-4989-a0d1-9bee40686af3/konnectivity-agent/0.log"
Jul 30 18:34:48.691303 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:48.691270    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-203.ec2.internal_ebe8f1131fdea96935597faec1e85919/haproxy/0.log"
Jul 30 18:34:54.126488 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:54.126424    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-vgzwj_a5d7663e-d9ab-410e-9759-7477ba5c5f90/manager/0.log"
Jul 30 18:34:54.162413 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:54.162330    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-lblst_2c73c8dc-987b-4121-88f7-c7a90683936a/manager/0.log"
Jul 30 18:34:54.251161 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:54.251129    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-95lpf_5237a970-40b8-4b10-b4a1-237e27d0a2af/registry-server/0.log"
Jul 30 18:34:54.356231 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:54.356196    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-xmfjp_f07c466d-53a4-4416-94a3-4bef276fb42f/manager/0.log"
Jul 30 18:34:54.492080 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:54.491994    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-sjv69_73e3ca24-3939-4b09-89af-01b8100030cd/manager/0.log"
Jul 30 18:34:56.226485 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:56.226460    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_982e116f-d739-42f4-bfe6-cbb4f5ca6d1f/alertmanager/0.log"
Jul 30 18:34:56.250441 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:56.250412    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_982e116f-d739-42f4-bfe6-cbb4f5ca6d1f/config-reloader/0.log"
Jul 30 18:34:56.275002 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:56.274975    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_982e116f-d739-42f4-bfe6-cbb4f5ca6d1f/kube-rbac-proxy-web/0.log"
Jul 30 18:34:56.298032 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:56.298007    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_982e116f-d739-42f4-bfe6-cbb4f5ca6d1f/kube-rbac-proxy/0.log"
Jul 30 18:34:56.327126 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:56.327098    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_982e116f-d739-42f4-bfe6-cbb4f5ca6d1f/kube-rbac-proxy-metric/0.log"
Jul 30 18:34:56.350434 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:56.350397    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_982e116f-d739-42f4-bfe6-cbb4f5ca6d1f/prom-label-proxy/0.log"
Jul 30 18:34:56.374501 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:56.374467    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_982e116f-d739-42f4-bfe6-cbb4f5ca6d1f/init-config-reloader/0.log"
Jul 30 18:34:56.794082 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:56.794044    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xj656_fc79f2a5-6bc1-44a7-b5ef-5162a3a01940/node-exporter/0.log"
Jul 30 18:34:56.817039 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:56.817007    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xj656_fc79f2a5-6bc1-44a7-b5ef-5162a3a01940/kube-rbac-proxy/0.log"
Jul 30 18:34:56.839871 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:56.839846    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xj656_fc79f2a5-6bc1-44a7-b5ef-5162a3a01940/init-textfile/0.log"
Jul 30 18:34:57.193048 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:57.193019    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-566b6bf4dd-r8c5m_bb04c41e-aa36-409e-93e8-0e9abf1a764d/telemeter-client/0.log"
Jul 30 18:34:57.214118 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:57.214085    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-566b6bf4dd-r8c5m_bb04c41e-aa36-409e-93e8-0e9abf1a764d/reload/0.log"
Jul 30 18:34:57.241197 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:57.241166    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-566b6bf4dd-r8c5m_bb04c41e-aa36-409e-93e8-0e9abf1a764d/kube-rbac-proxy/0.log"
Jul 30 18:34:57.273490 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:57.273455    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f8c75f8bc-82jb9_036c7f90-b643-4f96-a932-37753f58d745/thanos-query/0.log"
Jul 30 18:34:57.297169 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:57.297143    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f8c75f8bc-82jb9_036c7f90-b643-4f96-a932-37753f58d745/kube-rbac-proxy-web/0.log"
Jul 30 18:34:57.319338 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:57.319311    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f8c75f8bc-82jb9_036c7f90-b643-4f96-a932-37753f58d745/kube-rbac-proxy/0.log"
Jul 30 18:34:57.343661 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:57.343632    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f8c75f8bc-82jb9_036c7f90-b643-4f96-a932-37753f58d745/prom-label-proxy/0.log"
Jul 30 18:34:57.366428 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:57.366395    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f8c75f8bc-82jb9_036c7f90-b643-4f96-a932-37753f58d745/kube-rbac-proxy-rules/0.log"
Jul 30 18:34:57.392173 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:57.392148    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f8c75f8bc-82jb9_036c7f90-b643-4f96-a932-37753f58d745/kube-rbac-proxy-metrics/0.log"
Jul 30 18:34:58.300031 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:58.299993    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp"]
Jul 30 18:34:58.306922 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:58.306896    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp"
Jul 30 18:34:58.314414 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:58.314385    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp"]
Jul 30 18:34:58.432347 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:58.432306    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432-sys\") pod \"perf-node-gather-daemonset-lmlhp\" (UID: \"4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp"
Jul 30 18:34:58.432625 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:58.432605    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432-podres\") pod \"perf-node-gather-daemonset-lmlhp\" (UID: \"4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp"
Jul 30 18:34:58.432805 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:58.432788    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2pp9\" (UniqueName: \"kubernetes.io/projected/4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432-kube-api-access-b2pp9\") pod \"perf-node-gather-daemonset-lmlhp\" (UID: \"4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp"
Jul 30 18:34:58.432990 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:58.432976    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432-lib-modules\") pod \"perf-node-gather-daemonset-lmlhp\" (UID: \"4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp"
Jul 30 18:34:58.433154 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:58.433132    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432-proc\") pod \"perf-node-gather-daemonset-lmlhp\" (UID: \"4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp"
Jul 30 18:34:58.534766 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:58.534342    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432-sys\") pod \"perf-node-gather-daemonset-lmlhp\" (UID: \"4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp"
Jul 30 18:34:58.534766 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:58.534396    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432-podres\") pod \"perf-node-gather-daemonset-lmlhp\" (UID: \"4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp"
Jul 30 18:34:58.534766 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:58.534426    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2pp9\" (UniqueName: \"kubernetes.io/projected/4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432-kube-api-access-b2pp9\") pod \"perf-node-gather-daemonset-lmlhp\" (UID: \"4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp"
Jul 30 18:34:58.534766 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:58.534464    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432-lib-modules\") pod \"perf-node-gather-daemonset-lmlhp\" (UID: \"4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp"
Jul 30 18:34:58.534766 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:58.534499    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432-proc\") pod \"perf-node-gather-daemonset-lmlhp\" (UID: \"4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp"
Jul 30 18:34:58.534766 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:58.534670    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432-proc\") pod \"perf-node-gather-daemonset-lmlhp\" (UID: \"4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp"
Jul 30 18:34:58.534766 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:58.534730    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432-sys\") pod \"perf-node-gather-daemonset-lmlhp\" (UID: \"4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp"
Jul 30 18:34:58.535252 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:58.534865    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432-podres\") pod \"perf-node-gather-daemonset-lmlhp\" (UID: \"4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp"
Jul 30 18:34:58.535252 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:58.535016    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432-lib-modules\") pod \"perf-node-gather-daemonset-lmlhp\" (UID: \"4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp"
Jul 30 18:34:58.543445 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:58.543419    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2pp9\" (UniqueName: \"kubernetes.io/projected/4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432-kube-api-access-b2pp9\") pod \"perf-node-gather-daemonset-lmlhp\" (UID: \"4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432\") " pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp"
Jul 30 18:34:58.625419 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:58.625326    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp"
Jul 30 18:34:58.834635 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:58.831552    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp"]
Jul 30 18:34:59.060728 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:59.060683    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp" event={"ID":"4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432","Type":"ContainerStarted","Data":"7595816cfe8a1762abea423846d33f4a44654d581acfb32ac41107581ac34dd6"}
Jul 30 18:34:59.061046 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:59.061024    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp" event={"ID":"4e5cbc76-6346-4d13-9dd0-1d5bd6d8e432","Type":"ContainerStarted","Data":"f1e6c747fd79229a6f4de8664bf480bfbaf1b044e941a55fccda7029cc9f5627"}
Jul 30 18:34:59.061988 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:59.061968    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp"
Jul 30 18:34:59.079027 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:59.078973    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp" podStartSLOduration=1.07895708 podStartE2EDuration="1.07895708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 18:34:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 18:34:59.077278712 +0000 UTC m=+4073.530842694" watchObservedRunningTime="2026-07-30 18:34:59.07895708 +0000 UTC m=+4073.532521070"
Jul 30 18:34:59.756470 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:34:59.756446    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-75449b9c89-dwplx_17bfb507-2a6d-4cfe-a87a-2c2d53fb8b77/console/0.log"
Jul 30 18:35:01.462291 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:01.462265    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5stgf_9efc23b9-3f92-42ef-b797-7a8608be85cd/dns/0.log"
Jul 30 18:35:01.487094 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:01.487062    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5stgf_9efc23b9-3f92-42ef-b797-7a8608be85cd/kube-rbac-proxy/0.log"
Jul 30 18:35:01.666510 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:01.666482    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mxb44_d9f46378-294e-4eda-a3c4-08073cac32f2/dns-node-resolver/0.log"
Jul 30 18:35:02.232672 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:02.232638    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5b7488fbb5-86nng_72f7ce7b-11e8-47a9-a9ae-a0cd782c46c8/registry/0.log"
Jul 30 18:35:02.283734 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:02.283700    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hkwdn_8267208d-1805-4410-bf0c-595b1f1604f1/node-ca/0.log"
Jul 30 18:35:03.599641 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:03.599607    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6ff87f57bc-md8k2_19d2cd9d-54a4-44d9-b08c-45c3afa59146/kube-auth-proxy/0.log"
Jul 30 18:35:03.777665 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:03.777639    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-t25kk_335be1a6-121e-41a2-b390-180e8c87ccdf/payload-processing/0.log"
Jul 30 18:35:04.651615 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:04.651578    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-brvhl_9feb0d2f-7be2-4f8a-b867-289f917ca8a4/serve-healthcheck-canary/0.log"
Jul 30 18:35:05.153648 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:05.153612    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-9pdgh_a7419876-6314-486e-b064-ee186d6cac15/insights-operator/0.log"
Jul 30 18:35:05.155146 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:05.155104    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-9pdgh_a7419876-6314-486e-b064-ee186d6cac15/insights-operator/1.log"
Jul 30 18:35:05.177994 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:05.177958    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-czd4t_c01212b7-d88c-4819-a9f2-48e7c8222e23/kube-rbac-proxy/0.log"
Jul 30 18:35:05.205281 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:05.205242    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-czd4t_c01212b7-d88c-4819-a9f2-48e7c8222e23/exporter/0.log"
Jul 30 18:35:05.231811 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:05.231782    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-czd4t_c01212b7-d88c-4819-a9f2-48e7c8222e23/extractor/0.log"
Jul 30 18:35:06.087582 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:06.087556    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vp2v8/perf-node-gather-daemonset-lmlhp"
Jul 30 18:35:07.700890 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:07.700783    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-mchct_b1596812-1f98-42ab-ace0-01fd85f11a4c/manager/0.log"
Jul 30 18:35:08.000214 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:08.000178    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-98m6c_421c5189-15a2-4509-9c61-2ffbfdd35204/manager/0.log"
Jul 30 18:35:08.207636 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:08.207201    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-59fc4686ff-gfw5x_170f7d7d-61e4-4269-8884-fae812916c3e/manager/0.log"
Jul 30 18:35:08.232145 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:08.231196    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-vxzgh_e86cc99c-4b17-4534-845a-73c8d130dba9/server/0.log"
Jul 30 18:35:08.325386 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:08.325304    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-jtrxh_29a8ca15-2df6-4ba4-adfc-a0024f4c3648/manager/0.log"
Jul 30 18:35:08.451196 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:08.451153    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-975dfc96f-6pz5c_9847f8a2-f792-433a-8e4e-25cad18e13d2/manager/0.log"
Jul 30 18:35:08.473953 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:08.473920    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-975dfc96f-6pz5c_9847f8a2-f792-433a-8e4e-25cad18e13d2/copy-manifests/0.log"
Jul 30 18:35:09.504952 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:09.504923    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6ff87f57bc-md8k2_19d2cd9d-54a4-44d9-b08c-45c3afa59146/kube-auth-proxy/0.log"
Jul 30 18:35:09.637867 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:09.637798    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-t25kk_335be1a6-121e-41a2-b390-180e8c87ccdf/payload-processing/0.log"
Jul 30 18:35:11.365305 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:11.365242    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6ff87f57bc-md8k2_19d2cd9d-54a4-44d9-b08c-45c3afa59146/kube-auth-proxy/0.log"
Jul 30 18:35:11.508434 ip-10-0-135-203 kubenswrapper[2577]: I0730 18:35:11.508403    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-t25kk_335be1a6-121e-41a2-b390-180e8c87ccdf/payload-processing/0.log"